{
  "cells": [
    {
      "attachments": {},
      "cell_type": "markdown",
      "metadata": {
        "id": "UMqBL77hMXP2"
      },
      "source": [
        "# Text classification pipeline using Deep Learning\n",
        "\n",
        "Authors:  \n",
        " - [Lior Gazit](https://www.linkedin.com/in/liorgazit).  \n",
        " - [Meysam Ghaffari](https://www.linkedin.com/in/meysam-ghaffari-ph-d-a2553088/).  \n",
        "\n",
        "This notebook is taught and reviewed in our book:  \n",
        "**[Mastering NLP from Foundations to LLMs](https://www.amazon.com/dp/1804619183)**  \n",
        "![image.png]()\n",
        "\n",
        "This Colab notebook is referenced in our book's Github repo:   \n",
        "https://github.com/PacktPublishing/Mastering-NLP-from-Foundations-to-LLMs   \n",
        "<a target=\"_blank\" href=\"https://colab.research.google.com/github/PacktPublishing/Mastering-NLP-from-Foundations-to-LLMs/blob/liors_branch/Chapter6_notebooks/Ch6_Text_Classification_DL.ipynb\">\n",
        "  <img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/>\n",
        "</a>"
      ]
    },
    {
      "attachments": {},
      "cell_type": "markdown",
      "metadata": {
        "id": "GoCA7atEspTg"
      },
      "source": [
        "**Objective: Processing conclusions of medical publication and inferring whether the conclusion represents an advice or not.**\n",
        "Based on Chapter 6, this notebook demonstrates a complete, end-to-end system design of a Deep Learning binary classifier.  \n",
        "Given a text observation, the classifier with infer whether the content of the text is a medical advice or not.  \n",
        "\n",
        "## The pipeline consists of:  \n",
        "1. Code settings  \n",
        "1. Gathering the data  \n",
        "1. Processing the data\n",
        "1. Prerocessing\n",
        "1. Preliminary data exploration  \n",
        "1. Feature engineering  \n",
        "1. Exploring the new numerical features\n",
        "1. Split to Train/Test\n",
        "1. Preliminary statistical analysis and feasibility study\n",
        "1. Feature selection\n",
        "1. Machine Learning  \n",
        " 11.1 Iterate over ML models  \n",
        " 11.2 Generate the chosen model  \n",
        " 11.3 Generating the train results  \n",
        " 11.4 Generating the test results  \n",
        "1. Deep Learning  \n",
        " 12.1 Formatting our data  \n",
        " 12.2 Training our LM  \n",
        "    12.2.1 Training hyperparameters  \n",
        "    12.2.2 Evaluation metric  \n",
        "    12.2.3 Trainer object  \n",
        "    12.2.4 Fine tuning  \n",
        "    12.2.5 Conclusion about training length  \n",
        "    12.2.6 Generating the train results   \n",
        "    12.2.7 Generating the test results   \n",
        "\n",
        "*Remark:  \n",
        "This is a complete ML pipeline that is designed to be fully inclusive in a single notebook file. This is meant to be an instruction tool. In a professional dev environment, the design should be distributed across reproducible `.py` files for reproducibility and efficiency.  \n",
        "\n",
        "## The Data:\n",
        "A data set from [Hugging Face:\n",
        "medical_meadow_health_advice](https://huggingface.co/datasets/medalpaca/medical_meadow_health_advice):   \n",
        ">>\n",
        "\"This dataset holds thousands of text observation which are conclusion of medical publications. Each conclusion is annotated as to whether it represents an advice or not.\"  \n",
        "\n",
        "**Requirements:**  \n",
        "* When running in Colab, use this runtime notebook setting: `Python 3, T4 GPU`  "
      ]
    },
    {
      "attachments": {},
      "cell_type": "markdown",
      "metadata": {
        "id": "g54Uf66Vz9Fi"
      },
      "source": [
        ">*```Disclaimer: The content and ideas presented in this notebook are solely those of the authors and do not represent the views or intellectual property of the authors' employers.```*"
      ]
    },
    {
      "attachments": {},
      "cell_type": "markdown",
      "metadata": {
        "id": "uJ1LYgXOpDJ6"
      },
      "source": [
        "Install:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "kO5KM7JCpDUP",
        "outputId": "03009d08-52fb-49e7-e904-df42b75b6c4c"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m7.0/7.0 MB\u001b[0m \u001b[31m24.3 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m536.7/536.7 kB\u001b[0m \u001b[31m48.9 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m143.3/143.3 kB\u001b[0m \u001b[31m18.1 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m622.8/622.8 kB\u001b[0m \u001b[31m50.2 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25h  Preparing metadata (setup.py) ... \u001b[?25l\u001b[?25hdone\n",
            "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m280.0/280.0 kB\u001b[0m \u001b[31m29.5 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m84.1/84.1 kB\u001b[0m \u001b[31m9.2 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m7.8/7.8 MB\u001b[0m \u001b[31m51.7 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m116.3/116.3 kB\u001b[0m \u001b[31m14.1 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m134.8/134.8 kB\u001b[0m \u001b[31m16.3 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25h  Preparing metadata (setup.py) ... \u001b[?25l\u001b[?25hdone\n",
            "  Building wheel for autocorrect (setup.py) ... \u001b[?25l\u001b[?25hdone\n",
            "  Building wheel for docopt (setup.py) ... \u001b[?25l\u001b[?25hdone\n"
          ]
        }
      ],
      "source": [
        "# REMARK:\n",
        "# If the below code error's out due to a Python package discrepency, it may be because new versions are causing it.\n",
        "# In which case, set \"default_installations\" to False to revert to the original image:\n",
        "default_installations = True\n",
        "if default_installations:\n",
        "  !pip -q install transformers==4.28.0 datasets num2words autocorrect accelerate evaluate\n",
        "else:\n",
        "  import requests\n",
        "  text_file_path = \"requirements__Ch6_Text_Classification_DL.txt\"\n",
        "  url = \"https://raw.githubusercontent.com/PacktPublishing/Mastering-NLP-from-Foundations-to-LLMs/main/Chapter6_notebooks/\" + text_file_path           \n",
        "  res = requests.get(url)\n",
        "  with open(text_file_path, \"w\") as f:\n",
        "    f.write(res.text)\n",
        "    \n",
        "  !pip install -r requirements__Ch6_Text_Classification_DL.txt"
      ]
    },
    {
      "attachments": {},
      "cell_type": "markdown",
      "metadata": {
        "id": "S_--320f-8Yi"
      },
      "source": [
        "Imports:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "E0zHJoN61Zu5",
        "outputId": "9464f3e2-4ac5-4d84-b06a-dc6a4ff23f67"
      },
      "outputs": [
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "[nltk_data] Downloading package punkt to /root/nltk_data...\n",
            "[nltk_data]   Unzipping tokenizers/punkt.zip.\n",
            "[nltk_data] Downloading package stopwords to /root/nltk_data...\n",
            "[nltk_data]   Unzipping corpora/stopwords.zip.\n",
            "[nltk_data] Downloading package wordnet to /root/nltk_data...\n"
          ]
        }
      ],
      "source": [
        "import numpy as np\n",
        "import pandas as pd\n",
        "import matplotlib\n",
        "import scipy\n",
        "import re\n",
        "from datasets import load_dataset, Dataset\n",
        "\n",
        "from num2words import num2words\n",
        "import nltk; nltk.download('punkt'); nltk.download('stopwords'); nltk.download('wordnet')\n",
        "from nltk.corpus import stopwords\n",
        "from nltk.stem.porter import PorterStemmer\n",
        "from nltk.stem import WordNetLemmatizer\n",
        "from autocorrect import Speller\n",
        "\n",
        "# ML imports:\n",
        "from sklearn.feature_extraction.text import TfidfVectorizer,CountVectorizer\n",
        "from sklearn.model_selection import cross_val_score, GridSearchCV, StratifiedKFold\n",
        "import sklearn.linear_model as lm\n",
        "from sklearn.ensemble import RandomForestClassifier\n",
        "from sklearn.neighbors import KNeighborsClassifier\n",
        "from sklearn.svm import SVC\n",
        "from sklearn.tree import DecisionTreeClassifier\n",
        "from sklearn.metrics import classification_report\n",
        "from sklearn.metrics import confusion_matrix\n",
        "\n",
        "# DL imports:\n",
        "from transformers import AutoTokenizer, TrainingArguments, Trainer\n",
        "from torch.utils.data import DataLoader\n",
        "import evaluate\n",
        "\n"
      ]
    },
    {
      "attachments": {},
      "cell_type": "markdown",
      "metadata": {
        "id": "Ql6clhha--CD"
      },
      "source": [
        "### Code Settings\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "02lniQAu-6ou"
      },
      "outputs": [],
      "source": [
        "# Items:\n",
        "# source_of_dataset: Whether to load the dataset from the package, or from a URL (for the particular dataset in this notebook, both options are available)\n",
        "# json_url: The url for the dataset's .json file\n",
        "# db_name: The db name from HuggingFace that holds the raw data\n",
        "# do_preprocessing: Logical, should preprocessing be performed\n",
        "# do_enhanced_preprocessing: Logical, should the computation-heavy preprocessing be performed\n",
        "# do_feature_eng: Logical\n",
        "# maximize_a_priori: Logocal, should the univariate preliminary feature selection be based on a priori or a postiori stats\n",
        "# num_chosen_features_per_class: Int, for the preliminary feature selection, how many features should be selected per class\n",
        "# test_size: ratio between 0 - 1\n",
        "# feature_eng_details: Either \"TfidfVectorizer\" (for TFIDF feature eng.) or \"CountVectorizer\" (for one hot encoding)\n",
        "# seed: Integer, the random seed used to insure reproducibility of results\n",
        "config_dict = {'source_of_dataset': \"json\",\n",
        "               'json_url': \"https://huggingface.co/datasets/medalpaca/medical_meadow_health_advice/raw/main/medical_meadow_health_advice.json\",\n",
        "               'db_name': \"medalpaca/medical_meadow_health_advice\",\n",
        "               'do_preprocessing': True,\n",
        "               'do_enhanced_preprocessing': False,\n",
        "               'do_feature_eng': True,\n",
        "               'maximize_a_priori': False,\n",
        "               'num_chosen_features_per_class': 200,\n",
        "               'test_size': 0.25,\n",
        "               'feature_eng_details': \"CountVectorizer-binary\",\n",
        "               'ngram_range_min': 1,\n",
        "               'ngram_range_max': 3,\n",
        "               'max_features': 1000,\n",
        "               'seed': 0}\n",
        "\n",
        "# Deep learning training parameters:\n",
        "# See description of input parameters in documentation for transformers.TrainingArguments.\n",
        "lm_training_args = TrainingArguments(\n",
        "    output_dir=\"test_trainer\",\n",
        "    num_train_epochs=2,\n",
        "    per_device_train_batch_size=8,\n",
        "    learning_rate=2e-5,\n",
        "    evaluation_strategy=\"steps\",\n",
        "    logging_steps=100,)\n",
        "\n",
        "layers_to_fine_tune = None\n",
        "\n",
        "pd.set_option('display.max_colwidth', None)\n",
        "pd.set_option('display.max_rows', None)\n"
      ]
    },
    {
      "attachments": {},
      "cell_type": "markdown",
      "metadata": {
        "id": "3d6x7m9s_eh0"
      },
      "source": [
        "### Gathering the Data"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "iJd1W79I4JmR"
      },
      "outputs": [],
      "source": [
        "# Taking the dataset either via the datasets package or directly from a URL:\n",
        "if config_dict[\"source_of_dataset\"] == \"datasets package\":\n",
        "  dataset_raw = load_dataset(config_dict[\"db_name\"])\n",
        "  dataset_df = pd.DataFrame(dataset_raw[\"train\"])\n",
        "elif config_dict[\"source_of_dataset\"] == \"json\":\n",
        "  # For this particualr dataset, the owners posted a .json file to accomodate the dataset:\n",
        "  dataset_df = pd.read_json(config_dict[\"json_url\"])\n"
      ]
    },
    {
      "attachments": {},
      "cell_type": "markdown",
      "metadata": {
        "id": "utLjs7A60MSQ"
      },
      "source": [
        "### Processing the Data\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "LVrk7NItpMmv"
      },
      "outputs": [],
      "source": [
        "dataset_df[\"_label_\"] = dataset_df[\"output\"]\n",
        "dataset_df[\"text\"] = dataset_df[\"input\"]\n",
        "dataset_df = dataset_df[[\"text\", \"_label_\"]]"
      ]
    },
    {
      "attachments": {},
      "cell_type": "markdown",
      "metadata": {
        "id": "gOo5mBx004k7"
      },
      "source": [
        "Let's have a quick look at the raw data:  "
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 244
        },
        "id": "L5wF9r9A04uA",
        "outputId": "fbdb74ee-4b74-471f-8e29-3a8074313382"
      },
      "outputs": [
        {
          "data": {
            "text/html": [
              "<style type=\"text/css\">\n",
              "#T_e6ead_row0_col0, #T_e6ead_row0_col1, #T_e6ead_row1_col0, #T_e6ead_row1_col1, #T_e6ead_row2_col0, #T_e6ead_row2_col1, #T_e6ead_row3_col0, #T_e6ead_row3_col1 {\n",
              "  text-align: left;\n",
              "}\n",
              "</style>\n",
              "<table id=\"T_e6ead\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr>\n",
              "      <th class=\"blank level0\" >&nbsp;</th>\n",
              "      <th id=\"T_e6ead_level0_col0\" class=\"col_heading level0 col0\" >text</th>\n",
              "      <th id=\"T_e6ead_level0_col1\" class=\"col_heading level0 col1\" >_label_</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th id=\"T_e6ead_level0_row0\" class=\"row_heading level0 row0\" >0</th>\n",
              "      <td id=\"T_e6ead_row0_col0\" class=\"data row0 col0\" >As we have previously shown an additional effect of vibrating insoles on improving static balance [3–5], it is possible that the combined effect on balance and gait will have a greater impact on falls and mobility than our data would imply. </td>\n",
              "      <td id=\"T_e6ead_row0_col1\" class=\"data row0 col1\" >This is no advice</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th id=\"T_e6ead_level0_row1\" class=\"row_heading level0 row1\" >1</th>\n",
              "      <td id=\"T_e6ead_row1_col0\" class=\"data row1 col0\" >Furthermore, gut microbiota analysis in mice treated with hypolipidemic drugs revealed a modification in composition in favor of probiotic-type bacteria from Lactobacillus spp. [49]. </td>\n",
              "      <td id=\"T_e6ead_row1_col1\" class=\"data row1 col1\" >This is no advice</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th id=\"T_e6ead_level0_row2\" class=\"row_heading level0 row2\" >2</th>\n",
              "      <td id=\"T_e6ead_row2_col0\" class=\"data row2 col0\" >Further research is recommended that may be helpful in establishing the direction of the association. </td>\n",
              "      <td id=\"T_e6ead_row2_col1\" class=\"data row2 col1\" >This is no advice</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th id=\"T_e6ead_level0_row3\" class=\"row_heading level0 row3\" >3</th>\n",
              "      <td id=\"T_e6ead_row3_col0\" class=\"data row3 col0\" >Further study will be necessary to test if postnatal intervention can effectively promote neurodevelopmental outcomes for healthy infants with below-average white matter development at birth. </td>\n",
              "      <td id=\"T_e6ead_row3_col1\" class=\"data row3 col1\" >This is no advice</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n"
            ],
            "text/plain": [
              "<pandas.io.formats.style.Styler at 0x7ac09eaad720>"
            ]
          },
          "execution_count": 6,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "dataset_df.head(4).style.set_properties(**{'text-align': 'left'})"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "pZ7hseIDJbX0",
        "outputId": "d7bc6699-3f12-44fa-b296-7ce0764975b5"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "Distribution of original labels:\n",
            "\n"
          ]
        },
        {
          "data": {
            "text/plain": [
              "_label_                \n",
              "This is no advice          6477\n",
              "This is a weak advice      1338\n",
              "This is a strong advice     861\n",
              "dtype: int64"
            ]
          },
          "execution_count": 7,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "print(\"Distribution of original labels:\\n\")\n",
        "dataset_df[[\"_label_\"]].value_counts()"
      ]
    },
    {
      "attachments": {},
      "cell_type": "markdown",
      "metadata": {
        "id": "uJIfEYzYzzC4"
      },
      "source": [
        "While one could train a multi-class classification model that would diffrentiate between all 3 labels, we will reduce this to to being a binary classification problem.  \n",
        "We are going to unite the two labels that indicate an advice so to have a binary decision: Whether a text represents and advice or not.  \n",
        "So we are going to re-label:  \n",
        ">>\n",
        "Label 0: **Not** an advice    \n",
        "Label 1: An advice\n",
        "\n",
        "So now the classification problem is a binary classification problem.  "
      ]
    },
    {
      "attachments": {},
      "cell_type": "markdown",
      "metadata": {
        "id": "AP7MrOXQ8Vas"
      },
      "source": [
        "#### Picking the class to be targeted"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "pbe_tsjmzzMX"
      },
      "outputs": [],
      "source": [
        "dataset_df_binary = dataset_df.copy()\n",
        "dataset_df_binary[\"_label_\"] = dataset_df[\"_label_\"].map({\"This is no advice\":0}).fillna(1).map(int)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "atVRw5FFIMaJ",
        "outputId": "c1a7f23d-2f2d-4d25-d2e8-d8b7d51ddc78"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "Distribution of new labels:\n",
            "\n"
          ]
        },
        {
          "data": {
            "text/plain": [
              "_label_\n",
              "0          6477\n",
              "1          2199\n",
              "dtype: int64"
            ]
          },
          "execution_count": 9,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "print(\"Distribution of new labels:\\n\")\n",
        "frequencies = dataset_df_binary[[\"_label_\"]].value_counts()\n",
        "frequencies"
      ]
    },
    {
      "attachments": {},
      "cell_type": "markdown",
      "metadata": {
        "id": "hiil7tVKv8kx"
      },
      "source": [
        "Recognizing the most frequent class.  \n",
        "This establish a baseline accuracy (a prior) for this dataset:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "Bs1Tv68QW1pA",
        "outputId": "50a7a83a-1fca-4809-f49e-814ba9552d8e"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "The most frequent class is: 0\n",
            "And its baseline accuracy is: 0.747\n"
          ]
        }
      ],
      "source": [
        "most_frequent_class = frequencies.index[:][0][0]\n",
        "print(\"The most frequent class is:\", most_frequent_class)\n",
        "print(\"And its baseline accuracy is:\", round((dataset_df_binary[\"_label_\"] == most_frequent_class).mean(), 3))"
      ]
    },
    {
      "attachments": {},
      "cell_type": "markdown",
      "metadata": {
        "id": "w5KVvs7J_giZ"
      },
      "source": [
        "### Preporcessing\n"
      ]
    },
    {
      "attachments": {},
      "cell_type": "markdown",
      "metadata": {
        "id": "RJm8NNust0H0"
      },
      "source": [
        "Define the preprocessing utility functions:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "uNJaNvEYt0SM"
      },
      "outputs": [],
      "source": [
        "def digits_to_words(match):\n",
        "  \"\"\"\n",
        "  Convert string digits to the English words. The function distinguishes between\n",
        "  cardinal and ordinal.\n",
        "  E.g. \"2\" becomes \"two\", while \"2nd\" becomes \"second\"\n",
        "\n",
        "  Input: str\n",
        "  Output: str\n",
        "  \"\"\"\n",
        "  suffixes = ['st', 'nd', 'rd', 'th']\n",
        "  # Making sure it's lower cased so not to rely on previous possible actions:\n",
        "  string = match[0].lower()\n",
        "  if string[-2:] in suffixes:\n",
        "    type='ordinal'\n",
        "    string = string[:-2]\n",
        "  else:\n",
        "    type='cardinal'\n",
        "\n",
        "  return num2words(string, to=type)\n",
        "\n",
        "\n",
        "def spelling_correction(text):\n",
        "    \"\"\"\n",
        "    Replace misspelled words with the correct spelling.\n",
        "\n",
        "    Input: str\n",
        "    Output: str\n",
        "    \"\"\"\n",
        "    corrector = Speller()\n",
        "    spells = [corrector(word) for word in text.split()]\n",
        "    return \" \".join(spells)\n",
        "\n",
        "\n",
        "def remove_stop_words(text):\n",
        "    \"\"\"\n",
        "    Remove stopwords.\n",
        "\n",
        "    Input: str\n",
        "    Output: str\n",
        "    \"\"\"\n",
        "    stopwords_set = set(stopwords.words('english'))\n",
        "    return \" \".join([word for word in text.split() if word not in stopwords_set])\n",
        "\n",
        "\n",
        "def stemming(text):\n",
        "    \"\"\"\n",
        "    Perform stemming of each word individually.\n",
        "\n",
        "    Input: str\n",
        "    Output: str\n",
        "    \"\"\"\n",
        "    stemmer = PorterStemmer()\n",
        "    return \" \".join([stemmer.stem(word) for word in text.split()])\n",
        "\n",
        "\n",
        "def lemmatizing(text):\n",
        "    \"\"\"\n",
        "    Perform lemmatization for each word individually.\n",
        "\n",
        "    Input: str\n",
        "    Output: str\n",
        "    \"\"\"\n",
        "    lemmatizer = WordNetLemmatizer()\n",
        "    return \" \".join([lemmatizer.lemmatize(word) for word in text.split()])\n",
        "\n",
        "\n",
        "\n",
        "\n",
        "def preprocessing(input_text):\n",
        "  \"\"\"\n",
        "  This function represents a complete pipeline for text preprocessing.\n",
        "\n",
        "  Input: str\n",
        "  Output: str\n",
        "  \"\"\"\n",
        "  output = input_text\n",
        "  # Lower casing:\n",
        "  output = output.lower()\n",
        "  # Convert digits to words:\n",
        "  # The following regex syntax looks for matching of consequtive digits tentatively followed by an ordinal suffix:\n",
        "  output = re.sub(r'\\d+(st)?(nd)?(rd)?(th)?', digits_to_words, output, flags=re.IGNORECASE)\n",
        "  # Remove punctuations and other special characters:\n",
        "  output = re.sub('[^ A-Za-z0-9]+', '', output)\n",
        "\n",
        "  if config_dict[\"do_enhanced_preprocessing\"]:\n",
        "    # Spelling corrections:\n",
        "    output = spelling_correction(output)\n",
        "\n",
        "  # Remove stop words:\n",
        "  output = remove_stop_words(output)\n",
        "\n",
        "  if config_dict[\"do_enhanced_preprocessing\"]:\n",
        "    # Stemming:\n",
        "    output = stemming(output)\n",
        "    # Lemmatizing:\n",
        "    output = lemmatizing(output)\n",
        "\n",
        "  return output"
      ]
    },
    {
      "attachments": {},
      "cell_type": "markdown",
      "metadata": {
        "id": "bgrWf0mEt0bm"
      },
      "source": [
        "Perform the preprocessing:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "IEedfizM6FtZ"
      },
      "outputs": [],
      "source": [
        "dataset_clean = dataset_df_binary.copy()\n",
        "if config_dict[\"do_preprocessing\"]:\n",
        "  dataset_clean[\"text\"] = [preprocessing(text) for text in dataset_clean[\"text\"]]"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 415
        },
        "id": "MSYLrwr7KREy",
        "outputId": "9ffe83db-b2cf-4ce8-cb13-3b358974b404"
      },
      "outputs": [
        {
          "data": {
            "text/html": [
              "<style type=\"text/css\">\n",
              "#T_2758d_row0_col0, #T_2758d_row0_col1, #T_2758d_row1_col0, #T_2758d_row1_col1, #T_2758d_row2_col0, #T_2758d_row2_col1, #T_2758d_row3_col0, #T_2758d_row3_col1, #T_2758d_row4_col0, #T_2758d_row4_col1, #T_2758d_row5_col0, #T_2758d_row5_col1, #T_2758d_row6_col0, #T_2758d_row6_col1, #T_2758d_row7_col0, #T_2758d_row7_col1, #T_2758d_row8_col0, #T_2758d_row8_col1, #T_2758d_row9_col0, #T_2758d_row9_col1 {\n",
              "  text-align: left;\n",
              "}\n",
              "</style>\n",
              "<table id=\"T_2758d\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr>\n",
              "      <th class=\"blank level0\" >&nbsp;</th>\n",
              "      <th id=\"T_2758d_level0_col0\" class=\"col_heading level0 col0\" >text</th>\n",
              "      <th id=\"T_2758d_level0_col1\" class=\"col_heading level0 col1\" >_label_</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th id=\"T_2758d_level0_row0\" class=\"row_heading level0 row0\" >0</th>\n",
              "      <td id=\"T_2758d_row0_col0\" class=\"data row0 col0\" >previously shown additional effect vibrating insoles improving static balance threefive possible combined effect balance gait greater impact falls mobility data would imply</td>\n",
              "      <td id=\"T_2758d_row0_col1\" class=\"data row0 col1\" >0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th id=\"T_2758d_level0_row1\" class=\"row_heading level0 row1\" >1</th>\n",
              "      <td id=\"T_2758d_row1_col0\" class=\"data row1 col0\" >furthermore gut microbiota analysis mice treated hypolipidemic drugs revealed modification composition favor probiotictype bacteria lactobacillus spp fortynine</td>\n",
              "      <td id=\"T_2758d_row1_col1\" class=\"data row1 col1\" >0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th id=\"T_2758d_level0_row2\" class=\"row_heading level0 row2\" >2</th>\n",
              "      <td id=\"T_2758d_row2_col0\" class=\"data row2 col0\" >research recommended may helpful establishing direction association</td>\n",
              "      <td id=\"T_2758d_row2_col1\" class=\"data row2 col1\" >0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th id=\"T_2758d_level0_row3\" class=\"row_heading level0 row3\" >3</th>\n",
              "      <td id=\"T_2758d_row3_col0\" class=\"data row3 col0\" >study necessary test postnatal intervention effectively promote neurodevelopmental outcomes healthy infants belowaverage white matter development birth</td>\n",
              "      <td id=\"T_2758d_row3_col1\" class=\"data row3 col1\" >0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th id=\"T_2758d_level0_row4\" class=\"row_heading level0 row4\" >4</th>\n",
              "      <td id=\"T_2758d_row4_col0\" class=\"data row4 col0\" >hand rheumatoid factor epstein barr virus infections described cause falsely elevated c trachomatis igm titers ten eleven</td>\n",
              "      <td id=\"T_2758d_row4_col1\" class=\"data row4 col1\" >0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th id=\"T_2758d_level0_row5\" class=\"row_heading level0 row5\" >5</th>\n",
              "      <td id=\"T_2758d_row5_col0\" class=\"data row5 col0\" >interestingly within context proposed environmental enrichment may actually mild stressor mice moncek et al two thousand four</td>\n",
              "      <td id=\"T_2758d_row5_col1\" class=\"data row5 col1\" >0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th id=\"T_2758d_level0_row6\" class=\"row_heading level0 row6\" >6</th>\n",
              "      <td id=\"T_2758d_row6_col0\" class=\"data row6 col0\" >thus failed quit reduction attempt marker increased decreased interest changing</td>\n",
              "      <td id=\"T_2758d_row6_col1\" class=\"data row6 col1\" >0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th id=\"T_2758d_level0_row7\" class=\"row_heading level0 row7\" >7</th>\n",
              "      <td id=\"T_2758d_row7_col0\" class=\"data row7 col0\" >study replicates prior findings hughes et al two thousand eight cannabis users trying change make many often rapid transitions among use usual reduction abstinence states b reduction attempts common abstinence attempts c quit reduction attempts shortlived participants achieve longterm abstinence use alcohol illicit drugs significantly increase abstinence e seek treatment</td>\n",
              "      <td id=\"T_2758d_row7_col1\" class=\"data row7 col1\" >0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th id=\"T_2758d_level0_row8\" class=\"row_heading level0 row8\" >8</th>\n",
              "      <td id=\"T_2758d_row8_col0\" class=\"data row8 col0\" >use crosssectional design limits ascertainment direction association ie possible mindfulness strategies may change symptoms social anxiety disappear strengthened mindfulness strategies may offer advantages could plausibly reduce symptoms social anxiety</td>\n",
              "      <td id=\"T_2758d_row8_col1\" class=\"data row8 col1\" >0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th id=\"T_2758d_level0_row9\" class=\"row_heading level0 row9\" >9</th>\n",
              "      <td id=\"T_2758d_row9_col0\" class=\"data row9 col0\" >fact tumor metastasis increased lungs eselectinoverexpressing mice reduced eselectin knockout mice eighteen</td>\n",
              "      <td id=\"T_2758d_row9_col1\" class=\"data row9 col1\" >0</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n"
            ],
            "text/plain": [
              "<pandas.io.formats.style.Styler at 0x7ac097380370>"
            ]
          },
          "execution_count": 13,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "dataset_clean[[\"text\", \"_label_\"]].head(10).style.set_properties(**{'text-align': 'left'})"
      ]
    },
    {
      "attachments": {},
      "cell_type": "markdown",
      "metadata": {
        "id": "Ht259WodQFLK"
      },
      "source": [
        "\n",
        "## Preliminary data exploration\n",
        "Every ML project should start with basic exploration of the data.  \n",
        "The objectives are mainly to explore the nature of the data, and to study whether there is a connection between the \"X\" data (in our case, the input text) and the desired value, \"Y\" (in our case, classifying as advice).  \n",
        "\n",
        "Here we start by looking at the most simple characteristic of the data: the length of each input.  \n",
        "We will later explore the statistical dependence between the language used and the advice label."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 582
        },
        "id": "zbe9QF-lNnLs",
        "outputId": "7854f0dd-dc72-4629-fd20-f24c92d7602f"
      },
      "outputs": [
        {
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAA2wAAAI1CAYAAACqmnnaAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjcuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/bCgiHAAAACXBIWXMAAA9hAAAPYQGoP6dpAABtuklEQVR4nO3deVyVdf7//+cBPAgqoCggikru5pomUS6ZGC5judSkYaIxWo2Wa5nTZC5TuJSpWVrzSdTJsmzUGisL99Ext9yn3FLJBDUREB2R5f37w5/n2xHc8BzOAR732+3cbl7v632u9+uSt+az67rel8UYYwQAAAAAcDseri4AAAAAAFAwAhsAAAAAuCkCGwAAAAC4KQIbAAAAALgpAhsAAAAAuCkCGwAAAAC4KQIbAAAAALgpAhsAAAAAuCkCGwAAAAC4KQIbgBJl/PjxslgsRTLWgw8+qAcffNC2vW7dOlksFn3++edFMv6AAQNUq1atIhmrsDIzM/WnP/1JISEhslgsGj58eJHXcO3PqSjVqlVLAwYMcMnYt2PAgAEqX768q8twOlfOBQAoLAIbALc1f/58WSwW26ds2bIKDQ1VdHS0Zs2apfPnzztknJMnT2r8+PHatWuXQ47nSO5c26144403NH/+fD333HP6xz/+oaeeeqrQxyruvxeudvHiRY0fP17r1q1zdSkAgNvg5eoCAOBmJk6cqPDwcGVnZyslJUXr1q3T8OHDNX36dH355Zdq2rSpre9f//pXvfzyy7d1/JMnT2rChAmqVauWmjdvfsvf++67725rnMK4UW1///vflZeX5/Qa7sSaNWt033336bXXXrvjY7nzz6k4uHjxoiZMmCBJXGUCgGKEwAbA7XXp0kWtWrWybY8dO1Zr1qzRH/7wBz3yyCP68ccf5ePjI0ny8vKSl5dz/2q7ePGifH19ZbVanTrOzZQpU8al49+K06dPq1GjRi4Z211+TnCMnJwc5eXl8fMEUOpwSySAYumhhx7Sq6++quPHj+ujjz6ytRf0DFtiYqLatGmjgIAAlS9fXvXr19df/vIXSVeeO7v33nslSQMHDrTdfjl//nxJV65ENG7cWDt27FC7du3k6+tr++71nofJzc3VX/7yF4WEhKhcuXJ65JFH9Msvv9j1ud6zTb8/5s1qK+gZtgsXLmjUqFEKCwuTt7e36tevrzfffFPGGLt+FotFQ4cO1fLly9W4cWN5e3vr7rvv1sqVKwv+Db/G6dOnFRcXp+DgYJUtW1bNmjXTggULbPuvPs939OhRffXVV7bajx07dt1jFtXP6Wptn332mV5//XVVr15dZcuWVceOHXX48OF8db377ru666675OPjo9atW+vf//73HT0LlZaWpuHDh9t+RnXq1NGUKVPsrpYeO3ZMFotFb775pj744APVrl1b3t7euvfee7Vt27Z8x1yyZIkaNWqksmXLqnHjxlq2bJnd/Dh27JiqVKkiSZowYYLt92/8+PF2x/n111/Vo0cPlS9fXlWqVNHo0aOVm5t703OqVauW/vCHP+i7775T8+bNVbZsWTVq1EhLly694/OfMWOG7fz/+9//3rCOjz76SK1bt5avr68qVqyodu3a3fAK6+XLlzVu3Di1bNlS/v7+KleunNq2bau1a9fm67t48WK1bNlSFSpUkJ+fn5o0aaKZM2fa9mdnZ2vChAmqW7euypYtq8DAQLVp00aJiYk3/f0DgBvhChuAYuupp57SX/7yF3333XcaNGhQgX3279+vP/zhD2ratKkmTpwob29vHT58WJs2bZIkNWzYUBMnTtS4ceM0ePBgtW3bVpJ0//33245x9uxZdenSRX369FG/fv0UHBx8w7pef/11WSwWjRkzRqdPn9aMGTMUFRWlXbt22a4E3opbqe33jDF65JFHtHbtWsXFxal58+b69ttv9eKLL+rXX3/V22+/bdd/48aNWrp0qf785z+rQoUKmjVrlnr37q2kpCQFBgZet67//e9/evDBB3X48GENHTpU4eHhWrJkiQYMGKC0tDQNGzZMDRs21D/+8Q+NGDFC1atX16hRoyTJFhqu5Yqf0+TJk+Xh4aHRo0crPT1dU6dOVUxMjLZs2WLrM2fOHA0dOlRt27bViBEjdOzYMfXo0UMVK1ZU9erVb3j8gly8eFHt27fXr7/+qmeeeUY1atTQf/7zH40dO1bJycmaMWOGXf+PP/5Y58+f1zPPPCOLxaKpU6eqV69e+vnnn21XWL/66is98cQTatKkieLj43Xu3DnFxcWpWrVqtuNUqVJFc+bM0XPPPaeePXuqV69ekmR3O3Fubq6io6MVERGhN998U6tWrdJbb72l2rVr67nnnrvpuR06dEhPPPGEnn32WcXGxiohIUGPP/64Vq5cqU6dOhXq/BMSEnTp0iUNHjxY3t7eqlSp0nXHnzBhgsaPH6/7779fEydOlNVq1ZYtW7RmzRo9/PDDBX4nIyND//d//6e+fftq0KBBOn/+vD788ENFR0dr69attltvExMT1bdvX3Xs2FFTpkyRJP3444/atGmThg0bJunK/yyKj4/Xn/70J7Vu3VoZGRnavn27fvjhB9v5A0ChGABwUwkJCUaS2bZt23X7+Pv7mxYtWti2X3vtNfP7v9refvttI8mcOXPmusfYtm2bkWQSEhLy7Wvfvr2RZObOnVvgvvbt29u2165daySZatWqmYyMDFv7Z599ZiSZmTNn2tpq1qxpYmNjb3rMG9UWGxtratasadtevny5kWT+9re/2fV77LHHjMViMYcPH7a1STJWq9Wubffu3UaSeeedd/KN9XszZswwksxHH31ka7t8+bKJjIw05cuXtzv3mjVrmm7dut3weMa45ufUsGFDk5WVZWufOXOmkWT27t1rjDEmKyvLBAYGmnvvvddkZ2fb+s2fP99Isjvm9Vz7c540aZIpV66cOXjwoF2/l19+2Xh6epqkpCRjjDFHjx41kkxgYKBJTU219fviiy+MJPOvf/3L1takSRNTvXp1c/78eVvbunXrjCS7+XHmzBkjybz22mv56oyNjTWSzMSJE+3aW7RoYVq2bHlL5ynJ/POf/7S1paenm6pVq9r9+bzd8/fz8zOnT5++6fiHDh0yHh4epmfPniY3N9duX15enu3X186FnJwcuzlgjDHnzp0zwcHB5umnn7a1DRs2zPj5+ZmcnJzr1tCsWbNbmusAcLu4JRJAsVa+fPkbrhYZEBAgSfriiy8KvUCHt7e3Bg4ceMv9+/fvrwoVKti2H3vsMVWtWlVff/11oca/VV9//bU8PT31wgsv2LWPGjVKxhh98803du1RUVGqXbu2bbtp06by8/PTzz//fNNxQkJC1LdvX1tbmTJl9MILLygzM1Pr16+/7dpd8XMaOHCg3fNQV6/aXT3/7du36+zZsxo0aJDdc5ExMTGqWLFioWpcsmSJ2rZtq4oVK+q3336zfaKiopSbm6sNGzbY9X/iiSfsxrq2xpMnT2rv3r3q37+/3bL87du3V5MmTW67vmeffdZuu23btjedD1eFhoaqZ8+etm0/Pz/1799fO3fuVEpKiqTbP//evXtf96rs7y1fvlx5eXkaN26cPDzs/2lzo9d8eHp62uZAXl6eUlNTlZOTo1atWumHH36w9QsICNCFCxdueHtjQECA9u/fr0OHDt20XgC4HQQ2AMVaZmamXTi61hNPPKEHHnhAf/rTnxQcHKw+ffros88+u61QUK1atdta6KBu3bp22xaLRXXq1Lnh81uOcPz4cYWGhub7/WjYsKFt/+/VqFEj3zEqVqyoc+fO3XScunXr5vuH8fXGuRWu+Dlde/5Xg9HV8796HnXq1LHr5+XlVej33x06dEgrV65UlSpV7D5RUVGSrjwb6Igar9d2I2XLls0Xjm5lPvx+vGvDUb169STJNvdv9/zDw8NvaewjR47Iw8OjUAvcLFiwQE2bNrU9d1alShV99dVXSk9Pt/X585//rHr16qlLly6qXr26nn766XzPe06cOFFpaWmqV6+emjRpohdffFF79uy57XoA4Fo8wwag2Dpx4oTS09Nv+A9THx8fbdiwQWvXrtVXX32llStX6tNPP9VDDz2k7777Tp6enjcd53aeO7tV1/u//rm5ubdUkyNcbxxzzQIlRcEVPydXnH9eXp46deqkl156qcD9VwPOVUVZY1HMu9s9f2f82fu9jz76SAMGDFCPHj304osvKigoSJ6enoqPj9eRI0ds/YKCgrRr1y59++23+uabb/TNN98oISFB/fv3ty22065dOx05ckRffPGFvvvuO/3f//2f3n77bc2dO1d/+tOfnHoeAEo2AhuAYusf//iHJCk6OvqG/Tw8PNSxY0d17NhR06dP1xtvvKFXXnlFa9euVVRU1A1vmSqMa2+JMsbo8OHDdgs8VKxYUWlpafm+e/z4cd1111227duprWbNmlq1apXOnz9vd5Xtp59+su13hJo1a2rPnj3Ky8uzu8p2p+MU9c/pZq6ex+HDh9WhQwdbe05Ojo4dO2b387xVtWvXVmZmpu2KkiNrvNa1bc7+/Tt8+LCMMXbjHDx4UJJsVyQdff5X1a5dW3l5efrvf/97W+/o+/zzz3XXXXdp6dKldnUX9N5Aq9Wq7t27q3v37srLy9Of//xnvf/++3r11Vdt/9OoUqVKGjhwoAYOHKjMzEy1a9dO48ePJ7ABuCPcEgmgWFqzZo0mTZqk8PBwxcTEXLdfampqvrar/6DLysqSJJUrV06SCgxQhbFw4UK75+o+//xzJScnq0uXLra22rVr6/vvv9fly5dtbStWrMi3/P/t1Na1a1fl5uZq9uzZdu1vv/22LBaL3fh3omvXrkpJSdGnn35qa8vJydE777yj8uXLq3379rd9TFf8nG6mVatWCgwM1N///nfl5OTY2hctWnTLtwle649//KM2b96sb7/9Nt++tLQ0u3FuRWhoqBo3bqyFCxcqMzPT1r5+/Xrt3bvXrq+vr69tHGc4efKkli1bZtvOyMjQwoUL1bx5c4WEhEhy/Plf1aNHD3l4eGjixIn5bqO90dXIq1cVf99ny5Yt2rx5s12/s2fP2m17eHjYAvvV+Xltn/Lly6tOnTq2/QBQWFxhA+D2vvnmG/3000/KycnRqVOntGbNGiUmJqpmzZr68ssvVbZs2et+d+LEidqwYYO6deummjVr6vTp03rvvfdUvXp1tWnTRtKV8BQQEKC5c+eqQoUKKleunCIiIm75+ZlrVapUSW3atNHAgQN16tQpzZgxQ3Xq1LF79cCf/vQnff755+rcubP++Mc/6siRI/roo4/sFgG53dq6d++uDh066JVXXtGxY8fUrFkzfffdd/riiy80fPjwfMcurMGDB+v999/XgAEDtGPHDtWqVUuff/65Nm3apBkzZtzwmcLrccXP6WasVqvGjx+v559/Xg899JD++Mc/6tixY5o/f75q165dqCtWL774or788kv94Q9/0IABA9SyZUtduHBBe/fu1eeff65jx46pcuXKt3XMN954Q48++qgeeOABDRw4UOfOndPs2bPVuHFjuxDn4+OjRo0a6dNPP1W9evVUqVIlNW7cWI0bN77t8yhIvXr1FBcXp23btik4OFjz5s3TqVOnlJCQ4NTzl648P/fKK69o0qRJatu2rXr16iVvb29t27ZNoaGhio+PL/B7f/jDH7R06VL17NlT3bp109GjRzV37lw1atTI7vfuT3/6k1JTU/XQQw+pevXqOn78uN555x01b97c9uxmo0aN9OCDD6ply5aqVKmStm/frs8//1xDhw697fMBADuuW6ASAG7s6rL+Vz9Wq9WEhISYTp06mZkzZ9otH3/Vtcv6r1692jz66KMmNDTUWK1WExoaavr27ZtvWfEvvvjCNGrUyHh5edktHd++fXtz9913F1jf9ZaL/+STT8zYsWNNUFCQ8fHxMd26dTPHjx/P9/233nrLVKtWzXh7e5sHHnjAbN++Pd8xb1Tbtcv6G2PM+fPnzYgRI0xoaKgpU6aMqVu3rpk2bZrd0ubGXFnWf8iQIflqut7rBq516tQpM3DgQFO5cmVjtVpNkyZNClxu/1aX9XfFz2nJkiV2/a4uJX/tecyaNcvUrFnTeHt7m9atW5tNmzaZli1bms6dO9/0vAr6/Tx//rwZO3asqVOnjrFaraZy5crm/vvvN2+++aa5fPmyXS3Tpk3Ld0wVsDT/4sWLTYMGDYy3t7dp3Lix+fLLL03v3r1NgwYN7Pr95z//MS1btjRWq9XuOLGxsaZcuXL5xrr2z9ONzrNbt27m22+/NU2bNjXe3t6mQYMG+X6PHXH+NzJv3jzTokUL4+3tbSpWrGjat29vEhMTbfuvnQt5eXnmjTfesP18W7RoYVasWJHvz9bnn39uHn74YRMUFGSsVqupUaOGeeaZZ0xycrKtz9/+9jfTunVrExAQYHx8fEyDBg3M66+/bjsnACgsizEueLocAIBiKi8vT1WqVFGvXr3097//3dXlXFfz5s1VpUqVGy5F7yi1atVS48aNtWLFCqePBQClDc+wAQBwHZcuXcr3DNTChQuVmpqqBx980DVFXSM7Ozvfs1/r1q3T7t273aZGAEDh8QwbAADX8f3332vEiBF6/PHHFRgYqB9++EEffvihGjdurMcff9zV5UmSfv31V0VFRalfv34KDQ3VTz/9pLlz5yokJCTfi7ABAMUPgQ0AgOuoVauWwsLCNGvWLKWmpqpSpUrq37+/Jk+efFsv6XamihUrqmXLlvq///s/nTlzRuXKlVO3bt00efJkBQYGuro8AMAd4hk2AAAAAHBTPMMGAAAAAG6KwAYAAAAAborABgAAAABuisAGAAAAAG6KwAYAAAAAborABgAAAABuisAGAAAAAG6KwAYAAAAAborABgAAAABuisAGAAAAAG6KwAYAAAAAborABgAAAABuisAGAAAAAG6KwAYAAAAAborABgAAAABuisAGAAAAAG6KwAYAAAAAborABgAAAABuisAGAAAAAG6KwAYAAAAAborABgDATWRlZWnMmDEKDQ2Vj4+PIiIilJiY6OqyAAClAIENAICbGDBggKZPn66YmBjNnDlTnp6e6tq1qzZu3Ojq0gAAJZzFGGNcXQQAAO5q69atioiI0LRp0zR69GhJ0qVLl9S4cWMFBQXpP//5j4srBACUZFxhAwDgBj7//HN5enpq8ODBtrayZcsqLi5Omzdv1i+//OLC6gAAJR2BDQCAG9i5c6fq1asnPz8/u/bWrVtLknbt2uWCqgAApQWBDQCAG0hOTlbVqlXztV9tO3nyZFGXBAAoRQhsAADcwP/+9z95e3vnay9btqxtPwAAzkJgAwDgBnx8fJSVlZWv/dKlS7b9AAA4C4ENAIAbqFq1qpKTk/O1X20LDQ0t6pIAAKUIgQ0AgBto3ry5Dh48qIyMDLv2LVu22PYDAOAsBDYAAG7gscceU25urj744ANbW1ZWlhISEhQREaGwsDAXVgcAKOm8XF0AAADuLCIiQo8//rjGjh2r06dPq06dOlqwYIGOHTumDz/80NXlAQBKOIsxxri6CAAA3NmlS5f06quv6qOPPtK5c+fUtGlTTZo0SdHR0a4uDQBQwhHYAAAAAMBN8QwbAAAAALgpAhsAAAAAuCkCGwAAAAC4KQIbAAAAALgpAhsAAAAAuCnew3YL8vLydPLkSVWoUEEWi8XV5QAAAABwEWOMzp8/r9DQUHl4OP/6F4HtFpw8eVJhYWGuLgMAAACAm/jll19UvXp1p49DYLsFFSpUkHTlh+Ln5+fiagAAAAC4SkZGhsLCwmwZwdkIbLfg6m2Qfn5+BDYAAAAARfaoFIuOAAAAAICbIrABAAAAgJsisAEAAACAm+IZNgAAAJRoubm5ys7OdnUZKEasVmuRLNl/KwhsAAAAKJGMMUpJSVFaWpqrS0Ex4+HhofDwcFmtVleXQmADAABAyXQ1rAUFBcnX17fIVvVD8ZaXl6eTJ08qOTlZNWrUcPm8IbABAACgxMnNzbWFtcDAQFeXg2KmSpUqOnnypHJyclSmTBmX1kJgg8O8nXjQpeOP6FTPpeMDAAD3cfWZNV9fXxdXguLo6q2Qubm5Lg9s7vEkHQAAAOAErr6dDcWTO80bAhsAAAAAuCkCGwAAAOBGHnzwQQ0fPtzVZUiS1q1bJ4vF4pSVNsePH6/g4GBZLBYtX77c4ccvKXiGDQAAAKVKUT93X1yes3/wwQfVvHlzzZgxw+lj/fjjj5owYYKWLVum++67TxUrVszXZ926derQoYPOnTungIAAh43trOM6C4ENAAAAQJE6cuSIJOnRRx91q+fF3BG3RAIAAABuLCsrS6NHj1a1atVUrlw5RUREaN26dbb98+fPV0BAgL799ls1bNhQ5cuXV+fOnZWcnGzrk5OToxdeeEEBAQEKDAzUmDFjFBsbqx49ekiSBgwYoPXr12vmzJmyWCyyWCw6duyY7fs7duxQq1at5Ovrq/vvv18HDhy4Yc179+7VQw89JB8fHwUGBmrw4MHKzMyUdOVWyO7du0u68oLqggLbsWPH1KFDB0lSxYoVZbFYNGDAAElX3pMWHx+v8PBw+fj4qFmzZvr8888lXXlZelRUlKKjo2WMkSSlpqaqevXqGjdu3A2P664IbAAAAIAbGzp0qDZv3qzFixdrz549evzxx9W5c2cdOnTI1ufixYt688039Y9//EMbNmxQUlKSRo8ebds/ZcoULVq0SAkJCdq0aZMyMjLsnhubOXOmIiMjNWjQICUnJys5OVlhYWG2/a+88oreeustbd++XV5eXnr66aevW++FCxcUHR2tihUratu2bVqyZIlWrVqloUOHSpJGjx6thIQESbKNda2wsDD985//lCQdOHBAycnJmjlzpiQpPj5eCxcu1Ny5c7V//36NGDFC/fr10/r162WxWLRgwQJt27ZNs2bNkiQ9++yzqlatmsaNG3fD47orbokEAAAA3FRSUpISEhKUlJSk0NBQSVcCz8qVK5WQkKA33nhD0pX3zs2dO1e1a9eWdCXkTZw40Xacd955R2PHjlXPnj0lSbNnz9bXX39t2+/v7y+r1SpfX1+FhITkq+P1119X+/btJUkvv/yyunXrpkuXLqls2bL5+n788ce6dOmSFi5cqHLlytnG6969u6ZMmaLg4GDbs2MFjSVJnp6eqlSpkiQpKCjI1j8rK0tvvPGGVq1apcjISEnSXXfdpY0bN+r9999X+/btVa1aNb3//vvq37+/UlJS9PXXX2vnzp3y8roSfQo6rjsjsAEAAABuau/evcrNzVW9evYLl2RlZSkwMNC27evrawtrklS1alWdPn1akpSenq5Tp06pdevWtv2enp5q2bKl8vLybqmOpk2b2h1bkk6fPq0aNWrk6/vjjz+qWbNmtrAmSQ888IDy8vJ04MABBQcH39KYBTl8+LAuXryoTp062bVfvnxZLVq0sG0//vjjWrZsmSZPnqw5c+aobt26hR7T1QhsAAAAgJvKzMyUp6enduzYIU9PT7t95cuXt/26TJkydvssFovtGS5H+P3xrz5zdqthz5GuPgf31VdfqVq1anb7vL29bb++ePGi7ffs97eOFkcENgAAAMBNtWjRQrm5uTp9+rTatm1bqGP4+/srODhY27ZtU7t27SRJubm5+uGHH9S8eXNbP6vVqtzc3DuuuWHDhpo/f74uXLhgu8q2adMmeXh4qH79+rd8HKvVaqv1qkaNGsnb21tJSUm2WzQLMmrUKHl4eOibb75R165d1a1bNz300EPXPa47Y9ERAAAAwE3Vq1dPMTEx6t+/v5YuXaqjR49q69atio+P11dffXXLx3n++ecVHx+vL774QgcOHNCwYcN07tw5uxUaa9WqpS1btujYsWP67bffCn0FLSYmRmXLllVsbKz27duntWvX6vnnn9dTTz11W7dD1qxZUxaLRStWrNCZM2eUmZmpChUqaPTo0RoxYoQWLFigI0eO6IcfftA777yjBQsWSLpy9W3evHlatGiROnXqpBdffFGxsbE6d+7cdY/rzghsAAAAgBtLSEhQ//79NWrUKNWvX189evTQtm3bCnx+7HrGjBmjvn37qn///oqMjFT58uUVHR1tt2jI6NGj5enpqUaNGqlKlSpKSkoqVL2+vr769ttvlZqaqnvvvVePPfaYOnbsqNmzZ9/WcapVq6YJEybo5ZdfVnBwsG2VyUmTJunVV19VfHy8GjZsqM6dO+urr75SeHi4zpw5o7i4OI0fP1733HOPJGnChAkKDg7Ws88+e8PjuiuLceTNrSVURkaG/P39lZ6eLj8/P1eXU6C3Ew+6ugSXG9Gp3s07AQCAUuHSpUs6evSowsPDC1zJsLTLy8tTw4YN9cc//lGTJk1ydTlu50bzp6izAc+wAQAAACXc8ePH9d1336l9+/bKysrS7NmzdfToUT355JOuLg03wS2RAAAAQAnn4eGh+fPn695779UDDzygvXv3atWqVWrYsKGrS8NNcIUNAAAAKOHCwsK0adMmV5eBQuAKGwAAAAC4KQIbAAAAALgpAhsAAABKrMK+SwylmzstpM8zbAAAAChxrFarPDw8dPLkSVWpUkVWq9XuJdHA9RhjdObMGVksFpUpU8bV5RDYAAAAUPJ4eHgoPDxcycnJOnnypKvLQTFjsVhUvXp1eXp6uroUAhsAAABKJqvVqho1aignJ0e5ubmuLgfFSJkyZdwirEkENgAAAJRgV29rc4db24DCYNERAAAAAHBTBDYAAAAAcFMENgAAAABwUwQ2AAAAAHBTBDYAAAAAcFMENgAAAABwUwQ2AAAAAHBTBDYAAAAAcFMENgAAAABwUwQ2AAAAAHBTBDYAAAAAcFMENgAAAABwUwQ2AAAAAHBTLg1sGzZsUPfu3RUaGiqLxaLly5fb7R8wYIAsFovdp3PnznZ9UlNTFRMTIz8/PwUEBCguLk6ZmZl2ffbs2aO2bduqbNmyCgsL09SpU519agAAAABwx1wa2C5cuKBmzZrp3XffvW6fzp07Kzk52fb55JNP7PbHxMRo//79SkxM1IoVK7RhwwYNHjzYtj8jI0MPP/ywatasqR07dmjatGkaP368PvjgA6edFwAAAAA4gpcrB+/SpYu6dOlywz7e3t4KCQkpcN+PP/6olStXatu2bWrVqpUk6Z133lHXrl315ptvKjQ0VIsWLdLly5c1b948Wa1W3X333dq1a5emT59uF+wAAAAAwN24/TNs69atU1BQkOrXr6/nnntOZ8+ete3bvHmzAgICbGFNkqKiouTh4aEtW7bY+rRr105Wq9XWJzo6WgcOHNC5c+cKHDMrK0sZGRl2HwAAAAAoam4d2Dp37qyFCxdq9erVmjJlitavX68uXbooNzdXkpSSkqKgoCC773h5ealSpUpKSUmx9QkODrbrc3X7ap9rxcfHy9/f3/YJCwtz9KkBAAAAwE259JbIm+nTp4/t102aNFHTpk1Vu3ZtrVu3Th07dnTauGPHjtXIkSNt2xkZGYQ2AAAAAEXOra+wXeuuu+5S5cqVdfjwYUlSSEiITp8+bdcnJydHqamptufeQkJCdOrUKbs+V7ev92yct7e3/Pz87D4AAAAAUNSKVWA7ceKEzp49q6pVq0qSIiMjlZaWph07dtj6rFmzRnl5eYqIiLD12bBhg7Kzs219EhMTVb9+fVWsWLFoTwAAAAAAboNLA1tmZqZ27dqlXbt2SZKOHj2qXbt2KSkpSZmZmXrxxRf1/fff69ixY1q9erUeffRR1alTR9HR0ZKkhg0bqnPnzho0aJC2bt2qTZs2aejQoerTp49CQ0MlSU8++aSsVqvi4uK0f/9+ffrpp5o5c6bdLY8AAAAA4I5cGti2b9+uFi1aqEWLFpKkkSNHqkWLFho3bpw8PT21Z88ePfLII6pXr57i4uLUsmVL/fvf/5a3t7ftGIsWLVKDBg3UsWNHde3aVW3atLF7x5q/v7++++47HT16VC1bttSoUaM0btw4lvQHAAAA4PYsxhjj6iLcXUZGhvz9/ZWenu62z7O9nXjQ1SW43IhO9VxdAgAAAEq4os4GxeoZNgAAAAAoTQhsAAAAAOCmCGwAAAAA4KYIbAAAAADgpghsAAAAAOCmCGwAAAAA4KYIbAAAAADgpghsAAAAAOCmCGwAAAAA4KYIbAAAAADgpghsAAAAAOCmCGwAAAAA4KYIbAAAAADgpghsAAAAAOCmChXYfv75Z0fXAQAAAAC4RqECW506ddShQwd99NFHunTpkqNrAgAAAACokIHthx9+UNOmTTVy5EiFhITomWee0datWx1dGwAAAACUaoUKbM2bN9fMmTN18uRJzZs3T8nJyWrTpo0aN26s6dOn68yZM46uEwAAAABKnTtadMTLy0u9evXSkiVLNGXKFB0+fFijR49WWFiY+vfvr+TkZEfVCQAAAACljtedfHn79u2aN2+eFi9erHLlymn06NGKi4vTiRMnNGHCBD366KPcKoki83biQZeOP6JTPZeODwAAgJKnUIFt+vTpSkhI0IEDB9S1a1ctXLhQXbt2lYfHlQt24eHhmj9/vmrVquXIWgEAAACgVClUYJszZ46efvppDRgwQFWrVi2wT1BQkD788MM7Kg4AAAAASrNCBbZDhw7dtI/ValVsbGxhDg8AAAAAUCEXHUlISNCSJUvytS9ZskQLFiy446IAAAAAAIUMbPHx8apcuXK+9qCgIL3xxht3XBQAAAAAoJCBLSkpSeHh4fnaa9asqaSkpDsuCgAAAABQyMAWFBSkPXv25GvfvXu3AgMD77goAAAAAEAhA1vfvn31wgsvaO3atcrNzVVubq7WrFmjYcOGqU+fPo6uEQAAAABKpUKtEjlp0iQdO3ZMHTt2lJfXlUPk5eWpf//+PMMGAAAAAA5SqMBmtVr16aefatKkSdq9e7d8fHzUpEkT1axZ09H1AQAAAECpVajAdlW9evVUr149R9UCAAAAAPidQgW23NxczZ8/X6tXr9bp06eVl5dnt3/NmjUOKQ4AAAAASrNCBbZhw4Zp/vz56tatmxo3biyLxeLougAAAACg1CtUYFu8eLE+++wzde3a1dH1AAAAAAD+f4Va1t9qtapOnTqOrgUAAAAA8DuFCmyjRo3SzJkzZYxxdD0AAAAAgP9foW6J3Lhxo9auXatvvvlGd999t8qUKWO3f+nSpQ4pDgAAAABKs0IFtoCAAPXs2dPRtQAAAAAAfqdQgS0hIcHRdQAAAAAArlGoZ9gkKScnR6tWrdL777+v8+fPS5JOnjypzMxMhxUHAAAAAKVZoa6wHT9+XJ07d1ZSUpKysrLUqVMnVahQQVOmTFFWVpbmzp3r6DoBAAAAoNQp1BW2YcOGqVWrVjp37px8fHxs7T179tTq1asdVhwAAAAAlGaFCmz//ve/9de//lVWq9WuvVatWvr1119v+TgbNmxQ9+7dFRoaKovFouXLl9vtN8Zo3Lhxqlq1qnx8fBQVFaVDhw7Z9UlNTVVMTIz8/PwUEBCguLi4fLdl7tmzR23btlXZsmUVFhamqVOn3t4JAwAAAIALFCqw5eXlKTc3N1/7iRMnVKFChVs+zoULF9SsWTO9++67Be6fOnWqZs2apblz52rLli0qV66coqOjdenSJVufmJgY7d+/X4mJiVqxYoU2bNigwYMH2/ZnZGTo4YcfVs2aNbVjxw5NmzZN48eP1wcffHAbZwwAAAAARc9iCvH26yeeeEL+/v764IMPVKFCBe3Zs0dVqlTRo48+qho1ahRqFUmLxaJly5apR48ekq5cXQsNDdWoUaM0evRoSVJ6erqCg4M1f/589enTRz/++KMaNWqkbdu2qVWrVpKklStXqmvXrjpx4oRCQ0M1Z84cvfLKK0pJSbFdEXz55Ze1fPly/fTTT7dUW0ZGhvz9/ZWeni4/P7/bPrei8HbiQVeXUOqN6FTP1SUAAADAyYo6GxTqCttbb72lTZs2qVGjRrp06ZKefPJJ2+2QU6ZMcUhhR48eVUpKiqKiomxt/v7+ioiI0ObNmyVJmzdvVkBAgC2sSVJUVJQ8PDy0ZcsWW5927drZ3b4ZHR2tAwcO6Ny5cwWOnZWVpYyMDLsPAAAAABS1Qq0SWb16de3evVuLFy/Wnj17lJmZqbi4OMXExNgtQnInUlJSJEnBwcF27cHBwbZ9KSkpCgoKstvv5eWlSpUq2fUJDw/Pd4yr+ypWrJhv7Pj4eE2YMMEh5wEAAAAAhVWowCZdCUb9+vVzZC1uY+zYsRo5cqRtOyMjQ2FhYS6sCAAAAEBpVKjAtnDhwhvu79+/f6GK+b2QkBBJ0qlTp1S1alVb+6lTp9S8eXNbn9OnT9t9LycnR6mpqbbvh4SE6NSpU3Z9rm5f7XMtb29veXt73/E5AAAAAMCdKFRgGzZsmN12dna2Ll68KKvVKl9fX4cEtvDwcIWEhGj16tW2gJaRkaEtW7boueeekyRFRkYqLS1NO3bsUMuWLSVJa9asUV5eniIiImx9XnnlFWVnZ6tMmTKSpMTERNWvX7/A2yEBAAAAwF0UatGRc+fO2X0yMzN14MABtWnTRp988sktHyczM1O7du3Srl27JF1ZaGTXrl1KSkqSxWLR8OHD9be//U1ffvml9u7dq/79+ys0NNS2kmTDhg3VuXNnDRo0SFu3btWmTZs0dOhQ9enTR6GhoZKkJ598UlarVXFxcdq/f78+/fRTzZw50+6WRwAAAABwR4V+hu1adevW1eTJk9WvX79bXi5/+/bt6tChg237aoiKjY3V/Pnz9dJLL+nChQsaPHiw0tLS1KZNG61cuVJly5a1fWfRokUaOnSoOnbsKA8PD/Xu3VuzZs2y7ff399d3332nIUOGqGXLlqpcubLGjRtn9642AAAAAHBHhXoP2/Xs2rVL7dq1K3HL4PMeNtwK3sMGAABQ8hV1NijUFbYvv/zSbtsYo+TkZM2ePVsPPPCAQwoDAAAAgNKuUIHt6jNkV1ksFlWpUkUPPfSQ3nrrLUfUBQAAAAClXqECW15enqPrAAAAAABco1CrRAIAAAAAnK9QV9huZ0n86dOnF2YIAAAAACj1ChXYdu7cqZ07dyo7O1v169eXJB08eFCenp665557bP0sFotjqgQAAACAUqhQga179+6qUKGCFixYoIoVK0q68jLtgQMHqm3btho1apRDiwQAAACA0qhQz7C99dZbio+Pt4U1SapYsaL+9re/sUokAAAAADhIoQJbRkaGzpw5k6/9zJkzOn/+/B0XBQAAAAAoZGDr2bOnBg4cqKVLl+rEiRM6ceKE/vnPfyouLk69evVydI0AAAAAUCoV6hm2uXPnavTo0XryySeVnZ195UBeXoqLi9O0adMcWiAAAAAAlFaFCmy+vr567733NG3aNB05ckSSVLt2bZUrV86hxQEAAABAaXZHL85OTk5WcnKy6tatq3LlyskY46i6AAAAAKDUK1RgO3v2rDp27Kh69eqpa9euSk5OliTFxcWxpD8AAAAAOEihAtuIESNUpkwZJSUlydfX19b+xBNPaOXKlQ4rDgAAAABKs0I9w/bdd9/p22+/VfXq1e3a69atq+PHjzukMAAAAAAo7Qp1he3ChQt2V9auSk1Nlbe39x0XBQAAAAAoZGBr27atFi5caNu2WCzKy8vT1KlT1aFDB4cVBwAAAAClWaFuiZw6dao6duyo7du36/Lly3rppZe0f/9+paamatOmTY6uEQAAAABKpUJdYWvcuLEOHjyoNm3a6NFHH9WFCxfUq1cv7dy5U7Vr13Z0jQAAAABQKt32Fbbs7Gx17txZc+fO1SuvvOKMmgAAAAAAKsQVtjJlymjPnj3OqAUAAAAA8DuFuiWyX79++vDDDx1dCwAAAADgdwq16EhOTo7mzZunVatWqWXLlipXrpzd/unTpzukOAAAAAAozW4rsP3888+qVauW9u3bp3vuuUeSdPDgQbs+FovFcdUBAAAAQCl2W4Gtbt26Sk5O1tq1ayVJTzzxhGbNmqXg4GCnFAcAAAAApdltPcNmjLHb/uabb3ThwgWHFgQAAAAAuKJQi45cdW2AAwAAAAA4zm0FNovFku8ZNZ5ZAwAAAADnuK1n2IwxGjBggLy9vSVJly5d0rPPPptvlcilS5c6rkIAAAAAKKVuK7DFxsbabffr18+hxQAAAAAA/p/bCmwJCQnOqgMAAAAAcI07WnQEAAAAAOA8BDYAAAAAcFMENgAAAABwUwQ2AAAAAHBTBDYAAAAAcFO3tUokgOt7O/GgS8cf0ameS8cHAACA43GFDQAAAADcFIENAAAAANwUgQ0AAAAA3BSBDQAAAADclFsHtvHjx8tisdh9GjRoYNt/6dIlDRkyRIGBgSpfvrx69+6tU6dO2R0jKSlJ3bp1k6+vr4KCgvTiiy8qJyenqE8FAAAAAG6b268Seffdd2vVqlW2bS+v/1fyiBEj9NVXX2nJkiXy9/fX0KFD1atXL23atEmSlJubq27duikkJET/+c9/lJycrP79+6tMmTJ64403ivxcAAAAAOB2uH1g8/LyUkhISL729PR0ffjhh/r444/10EMPSZISEhLUsGFDff/997rvvvv03Xff6b///a9WrVql4OBgNW/eXJMmTdKYMWM0fvx4Wa3Woj4dAAAAALhlbn1LpCQdOnRIoaGhuuuuuxQTE6OkpCRJ0o4dO5Sdna2oqChb3wYNGqhGjRravHmzJGnz5s1q0qSJgoODbX2io6OVkZGh/fv3X3fMrKwsZWRk2H0AAAAAoKi5dWCLiIjQ/PnztXLlSs2ZM0dHjx5V27Ztdf78eaWkpMhqtSogIMDuO8HBwUpJSZEkpaSk2IW1q/uv7rue+Ph4+fv72z5hYWGOPTEAAAAAuAVufUtkly5dbL9u2rSpIiIiVLNmTX322Wfy8fFx2rhjx47VyJEjbdsZGRmENgAAAABFzq2vsF0rICBA9erV0+HDhxUSEqLLly8rLS3Nrs+pU6dsz7yFhITkWzXy6nZBz8Vd5e3tLT8/P7sPAAAAABS1YhXYMjMzdeTIEVWtWlUtW7ZUmTJltHr1atv+AwcOKCkpSZGRkZKkyMhI7d27V6dPn7b1SUxMlJ+fnxo1alTk9QMAAADA7XDrWyJHjx6t7t27q2bNmjp58qRee+01eXp6qm/fvvL391dcXJxGjhypSpUqyc/PT88//7wiIyN13333SZIefvhhNWrUSE899ZSmTp2qlJQU/fWvf9WQIUPk7e3t4rMDAAAAgBtz68B24sQJ9e3bV2fPnlWVKlXUpk0bff/996pSpYok6e2335aHh4d69+6trKwsRUdH67333rN939PTUytWrNBzzz2nyMhIlStXTrGxsZo4caKrTgkAAAAAbpnFGGNcXYS7y8jIkL+/v9LT0932eba3Ew+6ugS42IhO9VxdAgAAQIlX1NmgWD3DBgAAAAClCYENAAAAANwUgQ0AAAAA3JRbLzpSnPAMGQAAAABH4wobAAAAALgpAhsAAAAAuCkCGwAAAAC4KQIbAAAAALgpAhsAAAAAuCkCGwAAAAC4KQIbAAAAALgpAhsAAAAAuCkCGwAAAAC4KQIbAAAAALgpAhsAAAAAuCkCGwAAAAC4KQIbAAAAALgpL1cXAMAx3k486NLxR3Sq59LxAQAASiKusAEAAACAmyKwAQAAAICbIrABAAAAgJsisAEAAACAmyKwAQAAAICbIrABAAAAgJsisAEAAACAmyKwAQAAAICbIrABAAAAgJsisAEAAACAmyKwAQAAAICbIrABAAAAgJsisAEAAACAm/JydQEASoa3Ew+6ugSN6FTP1SUAAAA4FFfYAAAAAMBNEdgAAAAAwE0R2AAAAADATRHYAAAAAMBNEdgAAAAAwE0R2AAAAADATbGsP4ASw9WvFuC1AgAAwNG4wgYAAAAAborABgAAAABuqlQFtnfffVe1atVS2bJlFRERoa1bt7q6JAAAAAC4rlLzDNunn36qkSNHau7cuYqIiNCMGTMUHR2tAwcOKCgoyNXlASgBeIYOAAA4msUYY1xdRFGIiIjQvffeq9mzZ0uS8vLyFBYWpueff14vv/zyDb+bkZEhf39/paeny8/Pr8A+rv6HGgCUdgRWAEBRuJVs4Eil4grb5cuXtWPHDo0dO9bW5uHhoaioKG3evDlf/6ysLGVlZdm209PTJV354VzPpQuZDqwYAHC74pf/4NLxhzxUx6XjAwCKxtVMUFTXvUpFYPvtt9+Um5ur4OBgu/bg4GD99NNP+frHx8drwoQJ+drDwsKcViMAoHj7i6sLAAAUqbNnz8rf39/p45SKwHa7xo4dq5EjR9q209LSVLNmTSUlJRXJDwWlV0ZGhsLCwvTLL78UySV2lF7MNRQV5hqKCnMNRSU9PV01atRQpUqVimS8UhHYKleuLE9PT506dcqu/dSpUwoJCcnX39vbW97e3vna/f39+QsARcLPz4+5hiLBXENRYa6hqDDXUFQ8PIpmwf1Ssay/1WpVy5YttXr1altbXl6eVq9ercjISBdWBgAAAADXVyqusEnSyJEjFRsbq1atWql169aaMWOGLly4oIEDB7q6NAAAAAAoUKkJbE888YTOnDmjcePGKSUlRc2bN9fKlSvzLURSEG9vb7322msF3iYJOBJzDUWFuYaiwlxDUWGuoagU9VwrNe9hAwAAAIDiplQ8wwYAAAAAxRGBDQAAAADcFIENAAAAANwUgQ0AAAAA3BSBDQAAAADcFIHtFrz77ruqVauWypYtq4iICG3dutXVJaEYiY+P17333qsKFSooKChIPXr00IEDB+z6XLp0SUOGDFFgYKDKly+v3r1769SpU3Z9kpKS1K1bN/n6+iooKEgvvviicnJyivJUUMxMnjxZFotFw4cPt7Ux1+Aov/76q/r166fAwED5+PioSZMm2r59u22/MUbjxo1T1apV5ePjo6ioKB06dMjuGKmpqYqJiZGfn58CAgIUFxenzMzMoj4VuLHc3Fy9+uqrCg8Pl4+Pj2rXrq1Jkybp94ucM9dQGBs2bFD37t0VGhoqi8Wi5cuX2+131Lzas2eP2rZtq7JlyyosLExTp069/WINbmjx4sXGarWaefPmmf3795tBgwaZgIAAc+rUKVeXhmIiOjraJCQkmH379pldu3aZrl27mho1apjMzExbn2effdaEhYWZ1atXm+3bt5v77rvP3H///bb9OTk5pnHjxiYqKsrs3LnTfP3116Zy5cpm7NixrjglFANbt241tWrVMk2bNjXDhg2ztTPX4AipqammZs2aZsCAAWbLli3m559/Nt9++605fPiwrc/kyZONv7+/Wb58udm9e7d55JFHTHh4uPnf//5n69O5c2fTrFkz8/3335t///vfpk6dOqZv376uOCW4qddff90EBgaaFStWmKNHj5olS5aY8uXLm5kzZ9r6MNdQGF9//bV55ZVXzNKlS40ks2zZMrv9jphX6enpJjg42MTExJh9+/aZTz75xPj4+Jj333//tmolsN1E69atzZAhQ2zbubm5JjQ01MTHx7uwKhRnp0+fNpLM+vXrjTHGpKWlmTJlypglS5bY+vz4449Gktm8ebMx5spfKh4eHiYlJcXWZ86cOcbPz89kZWUV7QnA7Z0/f97UrVvXJCYmmvbt29sCG3MNjjJmzBjTpk2b6+7Py8szISEhZtq0aba2tLQ04+3tbT755BNjjDH//e9/jSSzbds2W59vvvnGWCwW8+uvvzqveBQr3bp1M08//bRdW69evUxMTIwxhrkGx7g2sDlqXr333numYsWKdv/9HDNmjKlfv/5t1cctkTdw+fJl7dixQ1FRUbY2Dw8PRUVFafPmzS6sDMVZenq6JKlSpUqSpB07dig7O9tunjVo0EA1atSwzbPNmzerSZMmCg4OtvWJjo5WRkaG9u/fX4TVozgYMmSIunXrZjenJOYaHOfLL79Uq1at9PjjjysoKEgtWrTQ3//+d9v+o0ePKiUlxW6u+fv7KyIiwm6uBQQEqFWrVrY+UVFR8vDw0JYtW4ruZODW7r//fq1evVoHDx6UJO3evVsbN25Uly5dJDHX4ByOmlebN29Wu3btZLVabX2io6N14MABnTt37pbr8brTEyrJfvvtN+Xm5tr9w0WSgoOD9dNPP7moKhRneXl5Gj58uB544AE1btxYkpSSkiKr1aqAgAC7vsHBwUpJSbH1KWgeXt0HXLV48WL98MMP2rZtW759zDU4ys8//6w5c+Zo5MiR+stf/qJt27bphRdekNVqVWxsrG2uFDSXfj/XgoKC7PZ7eXmpUqVKzDXYvPzyy8rIyFCDBg3k6emp3Nxcvf7664qJiZEk5hqcwlHzKiUlReHh4fmOcXVfxYoVb6keAhtQhIYMGaJ9+/Zp48aNri4FJdAvv/yiYcOGKTExUWXLlnV1OSjB8vLy1KpVK73xxhuSpBYtWmjfvn2aO3euYmNjXVwdSpLPPvtMixYt0scff6y7775bu3bt0vDhwxUaGspcQ6nBLZE3ULlyZXl6euZbQe3UqVMKCQlxUVUoroYOHaoVK1Zo7dq1ql69uq09JCREly9fVlpaml3/38+zkJCQAufh1X2AdOWWx9OnT+uee+6Rl5eXvLy8tH79es2aNUteXl4KDg5mrsEhqlatqkaNGtm1NWzYUElJSZL+31y50X8/Q0JCdPr0abv9OTk5Sk1NZa7B5sUXX9TLL7+sPn36qEmTJnrqqac0YsQIxcfHS2KuwTkcNa8c9d9UAtsNWK1WtWzZUqtXr7a15eXlafXq1YqMjHRhZShOjDEaOnSoli1bpjVr1uS7NN6yZUuVKVPGbp4dOHBASUlJtnkWGRmpvXv32v3FkJiYKD8/v3z/aELp1bFjR+3du1e7du2yfVq1aqWYmBjbr5lrcIQHHngg3+tJDh48qJo1a0qSwsPDFRISYjfXMjIytGXLFru5lpaWph07dtj6rFmzRnl5eYqIiCiCs0BxcPHiRXl42P9z1dPTU3l5eZKYa3AOR82ryMhIbdiwQdnZ2bY+iYmJql+//i3fDimJZf1vZvHixcbb29vMnz/f/Pe//zWDBw82AQEBdiuoATfy3HPPGX9/f7Nu3TqTnJxs+1y8eNHW59lnnzU1atQwa9asMdu3bzeRkZEmMjLStv/qUusPP/yw2bVrl1m5cqWpUqUKS63jpn6/SqQxzDU4xtatW42Xl5d5/fXXzaFDh8yiRYuMr6+v+eijj2x9Jk+ebAICAswXX3xh9uzZYx599NECl8Ru0aKF2bJli9m4caOpW7cuS63DTmxsrKlWrZptWf+lS5eaypUrm5deesnWh7mGwjh//rzZuXOn2blzp5Fkpk+fbnbu3GmOHz9ujHHMvEpLSzPBwcHmqaeeMvv27TOLFy82vr6+LOvvDO+8846pUaOGsVqtpnXr1ub77793dUkoRiQV+ElISLD1+d///mf+/Oc/m4oVKxpfX1/Ts2dPk5ycbHecY8eOmS5duhgfHx9TuXJlM2rUKJOdnV3EZ4Pi5trAxlyDo/zrX/8yjRs3Nt7e3qZBgwbmgw8+sNufl5dnXn31VRMcHGy8vb1Nx44dzYEDB+z6nD171vTt29eUL1/e+Pn5mYEDB5rz588X5WnAzWVkZJhhw4aZGjVqmLJly5q77rrLvPLKK3bLpDPXUBhr164t8N9nsbGxxhjHzavdu3ebNm3aGG9vb1OtWjUzefLk267VYszvXhUPAADsZGZmatq0adqyZYu2bt2qc+fOKSEhQQMGDHB1aQCAUoBn2AAAuIHffvtNEydO1I8//qhmzZq5uhwAQCnDsv4AANxA1apVlZycrJCQEG3fvl333nuvq0sCAJQiXGEDAOAGvL29WfobAOAyBDYAAAAAcFMENgAAAABwUwQ2AAAAAHBTBDYAAAAAcFMENgAAAABwUwQ2AAAAAHBTBDYAAAAAcFO8OBsAgJuYPXu20tLSdPLkSUnSv/71L504cUKS9Pzzz8vf39+V5QEASjCLMca4uggAANxZrVq1dPz48QL3HT16VLVq1SraggAApQaBDQAAAADcFM+wAQAAAICbIrABAAAAgJsisAEAAACAmyKwAQAAAICbIrABAAAAgJviPWy3IC8vTydPnlSFChVksVhcXQ4AAAAAFzHG6Pz58woNDZWHh/OvfxHYbsHJkycVFhbm6jIAAAAAuIlffvlF1atXd/o4BLZbUKFCBUlXfih+fn4urgYAAACAq2RkZCgsLMyWEZyNwHYLrt4G6efnR2ADAAAAUGSPSrHoCAAAAAC4KQIbAAAAALgpAhsAAAAAuCmeYQMAAECJlpubq+zsbFeXgWLEarUWyZL9t4LABgAAgBLJGKOUlBSlpaW5uhQUMx4eHgoPD5fVanV1KQQ2AAAAlExXw1pQUJB8fX2LbFU/FG95eXk6efKkkpOTVaNGDZfPGwIbAAAASpzc3FxbWAsMDHR1OShmqlSpopMnTyonJ0dlypRxaS0ENtyxtxMPFtg+olO9Iq4EAADgiqvPrPn6+rq4EhRHV2+FzM3NdXlgc48n6QAAAAAncPXtbCie3GneENgAAAAAwE0R2AAAAAA38uCDD2r48OGuLkOStG7dOlksFqestDl+/HgFBwfLYrFo+fLlDj9+ScEzbAAAAChVrvf8vbMUl+f6H3zwQTVv3lwzZsxw+lg//vijJkyYoGXLlum+++5TxYoV8/VZt26dOnTooHPnzikgIMBhYzvruM5CYAMAAABQpI4cOSJJevTRR93qeTF3xC2RAAAAgBvLysrS6NGjVa1aNZUrV04RERFat26dbf/8+fMVEBCgb7/9Vg0bNlT58uXVuXNnJScn2/rk5OTohRdeUEBAgAIDAzVmzBjFxsaqR48ekqQBAwZo/fr1mjlzpiwWiywWi44dO2b7/o4dO9SqVSv5+vrq/vvv14EDB25Y8969e/XQQw/Jx8dHgYGBGjx4sDIzMyVduRWye/fukq68oLqgwHbs2DF16NBBklSxYkVZLBYNGDBA0pX3pMXHxys8PFw+Pj5q1qyZPv/8c0lXXpYeFRWl6OhoGWMkSampqapevbrGjRt3w+O6KwIbAAAA4MaGDh2qzZs3a/HixdqzZ48ef/xxde7cWYcOHbL1uXjxot5880394x//0IYNG5SUlKTRo0fb9k+ZMkWLFi1SQkKCNm3apIyMDLvnxmbOnKnIyEgNGjRIycnJSk5OVlhYmG3/K6+8orfeekvbt2+Xl5eXnn766evWe+HCBUVHR6tixYratm2blixZolWrVmno0KGSpNGjRyshIUGSbGNdKywsTP/85z8lSQcOHFBycrJmzpwpSYqPj9fChQs1d+5c7d+/XyNGjFC/fv20fv16WSwWLViwQNu2bdOsWbMkSc8++6yqVaumcePG3fC47opbIgEAAAA3lZSUpISEBCUlJSk0NFTSlcCzcuVKJSQk6I033pB05b1zc+fOVe3atSVdCXkTJ060Heedd97R2LFj1bNnT0nS7Nmz9fXXX9v2+/v7y2q1ytfXVyEhIfnqeP3119W+fXtJ0ssvv6xu3brp0qVLKlu2bL6+H3/8sS5duqSFCxeqXLlytvG6d++uKVOmKDg42PbsWEFjSZKnp6cqVaokSQoKCrL1z8rK0htvvKFVq1YpMjJSknTXXXdp48aNev/999W+fXtVq1ZN77//vvr376+UlBR9/fXX2rlzp7y8rkSfgo7rzghsAAAAgJvau3evcnNzVa+e/cIlWVlZCgwMtG37+vrawpokVa1aVadPn5Ykpaen69SpU2rdurVtv6enp1q2bKm8vLxbqqNp06Z2x5ak06dPq0aNGvn6/vjjj2rWrJktrEnSAw88oLy8PB04cEDBwcG3NGZBDh8+rIsXL6pTp0527ZcvX1aLFi1s248//riWLVumyZMna86cOapbt26hx3Q1AhsAAADgpjIzM+Xp6akdO3bI09PTbl/58uVtvy5TpozdPovFYnuGyxF+f/yrz5zdathzpKvPwX311VeqVq2a3T5vb2/bry9evGj7Pfv9raPFEYENAAAAcFMtWrRQbm6uTp8+rbZt2xbqGP7+/goODta2bdvUrl07SVJubq5++OEHNW/e3NbParUqNzf3jmtu2LCh5s+frwsXLtiusm3atEkeHh6qX7/+LR/HarXaar2qUaNG8vb2VlJSku0WzYKMGjVKHh4e+uabb9S1a1d169ZNDz300HWP687cftGRDRs2qHv37goNDS3wpXoDBgywrWRz9dO5c2e7PqmpqYqJiZGfn58CAgIUFxdnS+cAAACAu6pXr55iYmLUv39/LV26VEePHtXWrVsVHx+vr7766paP8/zzzys+Pl5ffPGFDhw4oGHDhuncuXN2KzTWqlVLW7Zs0bFjx/Tbb78V+gpaTEyMypYtq9jYWO3bt09r167V888/r6eeeuq2boesWbOmLBaLVqxYoTNnzigzM1MVKlTQ6NGjNWLECC1YsEBHjhzRDz/8oHfeeUcLFiyQdOXq27x587Ro0SJ16tRJL774omJjY3Xu3LnrHteduX1gu3Dhgpo1a6Z33333un2uLlt69fPJJ5/Y7Y+JidH+/fuVmJioFStWaMOGDRo8eLCzSwcAAADuWEJCgvr3769Ro0apfv366tGjh7Zt21bg82PXM2bMGPXt21f9+/dXZGSkypcvr+joaLtFQ0aPHi1PT081atRIVapUUVJSUqHq9fX11bfffqvU1FTde++9euyxx9SxY0fNnj37to5TrVo1TZgwQS+//LKCg4Ntq0xOmjRJr776quLj49WwYUN17txZX331lcLDw3XmzBnFxcVp/PjxuueeeyRJEyZMUHBwsJ599tkbHtddWYwjb251MovFomXLltneFyFducKWlpaW78rbVT/++KMaNWqkbdu2qVWrVpKklStXqmvXrjpx4oRttZ0bycjIkL+/v9LT0+Xn5+eIUylR3k48WGD7iE71CmwHAABwtkuXLuno0aMKDw8vcCXD0i4vL08NGzbUH//4R02aNMnV5bidG82fos4Gbn+F7VasW7dOQUFBql+/vp577jmdPXvWtm/z5s0KCAiwhTVJioqKkoeHh7Zs2VLg8bKyspSRkWH3AQAAAIqr48eP6+9//7sOHjyovXv36rnnntPRo0f15JNPuro03ESxD2ydO3fWwoULtXr1ak2ZMkXr169Xly5dbA8RpqSkKCgoyO47Xl5eqlSpklJSUgo8Znx8vPz9/W2f3780EAAAAChuPDw8NH/+fN1777164IEHtHfvXq1atUoNGzZ0dWm4iWK/SmSfPn1sv27SpImaNm2q2rVra926derYsWOhjjl27FiNHDnStp2RkUFoAwAAQLEVFhamTZs2uboMFEKxv8J2rbvuukuVK1fW4cOHJV15e/rVlwZelZOTo9TU1Ou+Wd3b21t+fn52HwAAAAAoaiUusJ04cUJnz561vYE9MjJSaWlp2rFjh63PmjVrlJeXp4iICFeVCQAAAAA35fa3RGZmZtqulknS0aNHtWvXLlWqVEmVKlXShAkT1Lt3b4WEhOjIkSN66aWXVKdOHUVHR0uSbanPQYMGae7cucrOztbQoUPVp0+fW1ohEoVX0OqRrBwJAACKUmHfJYbSzZ0W0nf7wLZ9+3Z16NDBtn312bLY2FjNmTNHe/bs0YIFC5SWlqbQ0FA9/PDDmjRpkry9vW3fWbRokYYOHaqOHTvKw8NDvXv31qxZs4r8XAAAAFA0rFarPDw8dPLkSVWpUkVWq9XuJdHA9RhjdObMGVksFpUpU8bV5RSv97C5Cu9hu7HrvYetIFxhAwAAReXy5ctKTk7WxYsXXV0KihmLxaLq1aurfPny+fYVdTZw+ytsAAAAQGFYrVbVqFFDOTk5tlc+AbeiTJky8vT0dHUZkghsAAAAKMGu3tbmDre2AYVR4laJBAAAAICSgsAGAAAAAG6KWyLhFq63cAmLlAAAAKA04wobAAAAALgpAhsAAAAAuCkCGwAAAAC4KQIbAAAAALgpAhsAAAAAuCkCGwAAAAC4KQIbAAAAALgpAhsAAAAAuCkCGwAAAAC4KQIbAAAAALgpAhsAAAAAuCkCGwAAAAC4KS9XFwD383biwQLbR3SqV8SVAAAAAKUbV9gAAAAAwE0R2AAAAADATbl9YNuwYYO6d++u0NBQWSwWLV++3LYvOztbY8aMUZMmTVSuXDmFhoaqf//+OnnypN0xatWqJYvFYveZPHlyEZ8JAAAAANwet3+G7cKFC2rWrJmefvpp9erVy27fxYsX9cMPP+jVV19Vs2bNdO7cOQ0bNkyPPPKItm/fbtd34sSJGjRokG27QoUKRVI/7F3v+TgAAAAA+bl9YOvSpYu6dOlS4D5/f38lJibatc2ePVutW7dWUlKSatSoYWuvUKGCQkJCnForAAAAADiS298SebvS09NlsVgUEBBg1z558mQFBgaqRYsWmjZtmnJycq57jKysLGVkZNh9AAAAAKCouf0Vtttx6dIljRkzRn379pWfn5+t/YUXXtA999yjSpUq6T//+Y/Gjh2r5ORkTZ8+vcDjxMfHa8KECUVVNgAAAAAUyGKMMa4u4lZZLBYtW7ZMPXr0yLcvOztbvXv31okTJ7Ru3Tq7wHatefPm6ZlnnlFmZqa8vb3z7c/KylJWVpZtOyMjQ2FhYUpPT7/hcUsKd3rOjHe/AQAAwJ1kZGTI39+/yLJBibjClp2drT/+8Y86fvy41qxZc9PfuIiICOXk5OjYsWOqX79+vv3e3t4FBjkAAAAAKErFPrBdDWuHDh3S2rVrFRgYeNPv7Nq1Sx4eHgoKCiqCCgEAAACgcNw+sGVmZurw4cO27aNHj2rXrl2qVKmSqlatqscee0w//PCDVqxYodzcXKWkpEiSKlWqJKvVqs2bN2vLli3q0KGDKlSooM2bN2vEiBHq16+fKlas6KrTwi0q6PZMbpMEAABAaeH2gW379u3q0KGDbXvkyJGSpNjYWI0fP15ffvmlJKl58+Z231u7dq0efPBBeXt7a/HixRo/fryysrIUHh6uESNG2I4DAAAAAO7K7QPbgw8+qButi3KzNVPuueceff/9944uC27oeoulcEUOAAAAxVWJew8bAAAAAJQUBDYAAAAAcFMENgAAAABwUwQ2AAAAAHBTBDYAAAAAcFNuv0okcK3rrQYJAAAAlDRcYQMAAAAAN0VgAwAAAAA3RWADAAAAADfltMD2888/O+vQAAAAAFAqOG3RkTp16qh9+/aKi4vTY489prJlyzprKNwBFvAAAAAA3JfTrrD98MMPatq0qUaOHKmQkBA988wz2rp1q7OGAwAAAIASx2mBrXnz5po5c6ZOnjypefPmKTk5WW3atFHjxo01ffp0nTlzxllDAwAAAECJ4PRFR7y8vNSrVy8tWbJEU6ZM0eHDhzV69GiFhYWpf//+Sk5OdnYJAAAAAFAsOT2wbd++XX/+859VtWpVTZ8+XaNHj9aRI0eUmJiokydP6tFHH3V2CQAAAABQLDlt0ZHp06crISFBBw4cUNeuXbVw4UJ17dpVHh5XMmJ4eLjmz5+vWrVqOasEAAAAACjWnBbY5syZo6effloDBgxQ1apVC+wTFBSkDz/80FklAAAAAECx5rTAdujQoZv2sVqtio2NdVYJAAAAAFCsOe0ZtoSEBC1ZsiRf+5IlS7RgwQJnDQsAAAAAJYbTAlt8fLwqV66crz0oKEhvvPGGs4YFAAAAgBLDaYEtKSlJ4eHh+dpr1qyppKSkWz7Ohg0b1L17d4WGhspisWj58uV2+40xGjdunKpWrSofHx9FRUXlux0zNTVVMTEx8vPzU0BAgOLi4pSZmVmo8wIAAACAouK0wBYUFKQ9e/bka9+9e7cCAwNv+TgXLlxQs2bN9O677xa4f+rUqZo1a5bmzp2rLVu2qFy5coqOjtalS5dsfWJiYrR//34lJiZqxYoV2rBhgwYPHnz7JwUAAAAARchpi4707dtXL7zwgipUqKB27dpJktavX69hw4apT58+t3ycLl26qEuXLgXuM8ZoxowZ+utf/2p7n9vChQsVHBys5cuXq0+fPvrxxx+1cuVKbdu2Ta1atZIkvfPOO+ratavefPNNhYaG3uGZwt29nXgwX9uITvVcUAkAAABwe5x2hW3SpEmKiIhQx44d5ePjIx8fHz388MN66KGHHPYM29GjR5WSkqKoqChbm7+/vyIiIrR582ZJ0ubNmxUQEGALa5IUFRUlDw8PbdmypcDjZmVlKSMjw+4DAAAAAEXNaVfYrFarPv30U02aNEm7d++Wj4+PmjRpopo1azpsjJSUFElScHCwXXtwcLBtX0pKioKCguz2e3l5qVKlSrY+14qPj9eECRMcVicAAAAAFIbTAttV9erVU716xev2s7Fjx2rkyJG27YyMDIWFhbmwojtX0G2BAAAAANyb0wJbbm6u5s+fr9WrV+v06dPKy8uz279mzZo7HiMkJESSdOrUKVWtWtXWfurUKTVv3tzW5/Tp03bfy8nJUWpqqu371/L29pa3t/cd1wcAAAAAd8Jpz7ANGzZMw4YNU25urho3bqxmzZrZfRwhPDxcISEhWr16ta0tIyNDW7ZsUWRkpCQpMjJSaWlp2rFjh63PmjVrlJeXp4iICIfUAQAAAADO4LQrbIsXL9Znn32mrl273tFxMjMzdfjwYdv20aNHtWvXLlWqVEk1atTQ8OHD9be//U1169ZVeHi4Xn31VYWGhqpHjx6SpIYNG6pz584aNGiQ5s6dq+zsbA0dOlR9+vRhhUgAAAAAbs2pi47UqVPnjo+zfft2dejQwbZ99dmy2NhYzZ8/Xy+99JIuXLigwYMHKy0tTW3atNHKlStVtmxZ23cWLVqkoUOHqmPHjvLw8FDv3r01a9asO64NAAAAAJzJYowxzjjwW2+9pZ9//lmzZ8+WxWJxxhBFJiMjQ/7+/kpPT5efn5+ryykUFh2xx3vYAAAAUBhFnQ2cdoVt48aNWrt2rb755hvdfffdKlOmjN3+pUuXOmtoAAAAACgRnBbYAgIC1LNnT2cdHgAAAABKPKcFtoSEBGcdGgAAAABKBact6y9ded/ZqlWr9P777+v8+fOSpJMnTyozM9OZwwIAAABAieC0K2zHjx9X586dlZSUpKysLHXq1EkVKlTQlClTlJWVpblz5zpraAAAAAAoEZz64uxWrVrp3Llz8vHxsbX37NnT7kXXAAAAAICCOe0K27///W/95z//kdVqtWuvVauWfv31V2cNCwAAAAAlhtOusOXl5Sk3Nzdf+4kTJ1ShQgVnDQsAAAAAJYbTAtvDDz+sGTNm2LYtFosyMzP12muvqWvXrs4aFgAAAABKDKfdEvnWW28pOjpajRo10qVLl/Tkk0/q0KFDqly5sj755BNnDQsAAAAAJYbTAlv16tW1e/duLV68WHv27FFmZqbi4uIUExNjtwgJAAAAAKBgTgtskuTl5aV+/fo5cwgAAAAAKLGcFtgWLlx4w/39+/d31tAAAAAAUCJYjDHGGQeuWLGi3XZ2drYuXrwoq9UqX19fpaamOmNYp8jIyJC/v7/S09Pl5+fn6nIK5e3Eg64uoVgY0ameq0sAAACAGyvqbOC0VSLPnTtn98nMzNSBAwfUpk0bFh0BAAAAgFvgtMBWkLp162ry5MkaNmxYUQ4LAAAAAMWSUxcdKXBALy+dPHmyqIcFbsn1bh3lVkkAAAC4gtMC25dffmm3bYxRcnKyZs+erQceeMBZwwIAAABAieG0wNajRw+7bYvFoipVquihhx7SW2+95axhSz0WFwEAAABKDqc9w5aXl2f3yc3NVUpKij7++GNVrVrVYePUqlVLFosl32fIkCGSpAcffDDfvmeffdZh4wMAAACAsxT5M2yOtm3bNuXm5tq29+3bp06dOunxxx+3tQ0aNEgTJ060bfv6+hZpjQAAAABQGE4LbCNHjrzlvtOnTy/0OFWqVLHbnjx5smrXrq327dvb2nx9fRUSElLoMQAAAADAFZwW2Hbu3KmdO3cqOztb9evXlyQdPHhQnp6euueee2z9LBaLw8a8fPmyPvroI40cOdLuuIsWLdJHH32kkJAQde/eXa+++uoNr7JlZWUpKyvLtp2RkeGwGgEAAADgVjktsHXv3l0VKlTQggULVLFiRUlXXqY9cOBAtW3bVqNGjXL4mMuXL1daWpoGDBhga3vyySdVs2ZNhYaGas+ePRozZowOHDigpUuXXvc48fHxmjBhgsPrAwAAAIDbYTHGGGccuFq1avruu+90991327Xv27dPDz/8sFPexRYdHS2r1ap//etf1+2zZs0adezYUYcPH1bt2rUL7FPQFbawsDClp6fLz8/P4XU7EqtEOgfvYQMAAIB0JRv4+/sXWTZw2hW2jIwMnTlzJl/7mTNndP78eYePd/z4ca1ateqGV84kKSIiQpJuGNi8vb3l7e3t8BoBAAAA4HY4bVn/nj17auDAgVq6dKlOnDihEydO6J///Kfi4uLUq1cvh4+XkJCgoKAgdevW7Yb9du3aJUkOfbUAAAAAADiD066wzZ07V6NHj9aTTz6p7OzsK4N5eSkuLk7Tpk1z6Fh5eXlKSEhQbGysvLz+3ykdOXJEH3/8sbp27arAwEDt2bNHI0aMULt27dS0aVOH1gAAAAAAjua0wObr66v33ntP06ZN05EjRyRJtWvXVrly5Rw+1qpVq5SUlKSnn37art1qtWrVqlWaMWOGLly4oLCwMPXu3Vt//etfHV4DAAAAADia01+cnZycrOTkZLVr104+Pj4yxjh0KX9Jevjhh1XQ2ilhYWFav369Q8cCAAAAgKLitGfYzp49q44dO6pevXrq2rWrkpOTJUlxcXFOWdIfAAAAAEoapwW2ESNGqEyZMkpKSrJ7SfUTTzyhlStXOmtYAAAAACgxnHZL5Hfffadvv/1W1atXt2uvW7eujh8/7qxhAQAAAKDEcNoVtgsXLthdWbsqNTWVd5wBAAAAwC1wWmBr27atFi5caNu2WCzKy8vT1KlT1aFDB2cNCwAAAAAlhtNuiZw6dao6duyo7du36/Lly3rppZe0f/9+paamatOmTc4aFgAAAABKDKddYWvcuLEOHjyoNm3a6NFHH9WFCxfUq1cv7dy5U7Vr13bWsAAAAABQYjjlClt2drY6d+6suXPn6pVXXnHGEECRejvxYL62EZ3quaASAAAAlCZOucJWpkwZ7dmzxxmHBgAAAIBSw2m3RPbr108ffvihsw4PAAAAACWe0xYdycnJ0bx587Rq1Sq1bNlS5cqVs9s/ffp0Zw0NAAAAACWCwwPbzz//rFq1amnfvn265557JEkHD9o//2OxWBw9LAAAAACUOA4PbHXr1lVycrLWrl0rSXriiSc0a9YsBQcHO3ooAAAAACjRHP4MmzHGbvubb77RhQsXHD0MAAAAAJR4Tlt05KprAxwAAAAA4NY4PLBZLJZ8z6jxzBoAAAAA3D6HP8NmjNGAAQPk7e0tSbp06ZKeffbZfKtELl261NFDAwAAAECJ4vDAFhsba7fdr18/Rw8BAAAAAKWCwwNbQkKCow8JAAAAAKWS0xcdAQAAAAAUjsOvsBW18ePHa8KECXZt9evX108//STpyjN0o0aN0uLFi5WVlaXo6Gi99957vBcOd+ztxIM37/Q7IzrVc1IlAAAAKKlKxBW2u+++W8nJybbPxo0bbftGjBihf/3rX1qyZInWr1+vkydPqlevXi6sFgAAAABuTbG/wiZJXl5eCgkJydeenp6uDz/8UB9//LEeeughSVeesWvYsKG+//573XfffUVdKgAAAADcshJxhe3QoUMKDQ3VXXfdpZiYGCUlJUmSduzYoezsbEVFRdn6NmjQQDVq1NDmzZuve7ysrCxlZGTYfQAAAACgqBX7K2wRERGaP3++6tevr+TkZE2YMEFt27bVvn37lJKSIqvVqoCAALvvBAcHKyUl5brHjI+Pz/dcnDu63WeoAAAAABQvxT6wdenSxfbrpk2bKiIiQjVr1tRnn30mHx+fQh1z7NixGjlypG07IyNDYWFhd1wrAAAAANyOEnFL5O8FBASoXr16Onz4sEJCQnT58mWlpaXZ9Tl16lSBz7xd5e3tLT8/P7sPAAAAABS1EhfYMjMzdeTIEVWtWlUtW7ZUmTJltHr1atv+AwcOKCkpSZGRkS6sEgAAAABurtjfEjl69Gh1795dNWvW1MmTJ/Xaa6/J09NTffv2lb+/v+Li4jRy5EhVqlRJfn5+ev755xUZGckKkQAAAADcXrEPbCdOnFDfvn119uxZValSRW3atNH333+vKlWqSJLefvtteXh4qHfv3nYvzgYAAAAAd2cxxhhXF+HuMjIy5O/vr/T0dLd6no1VIouXEZ3quboEAAAA3KGizgYl7hk2AAAAACgpCGwAAAAA4KYIbAAAAADgpghsAAAAAOCmCGwAAAAA4KYIbAAAAADgpghsAAAAAOCmCGwAAAAA4KYIbAAAAADgpghsAAAAAOCmCGwAAAAA4KYIbAAAAADgpghsAAAAAOCmCGwAAAAA4KYIbAAAAADgprxcXQBQWrydeDBf24hO9VxQCQAAAIoLAhvghgoKdxIBDwAAoLThlkgAAAAAcFMENgAAAABwUwQ2AAAAAHBTxf4Ztvj4eC1dulQ//fSTfHx8dP/992vKlCmqX7++rc+DDz6o9evX233vmWee0dy5c4u63EK53vNMAAAAAEq2Yn+Fbf369RoyZIi+//57JSYmKjs7Ww8//LAuXLhg12/QoEFKTk62faZOneqiigEAAADg1hT7K2wrV660254/f76CgoK0Y8cOtWvXztbu6+urkJCQoi4PAAAAAAqt2F9hu1Z6erokqVKlSnbtixYtUuXKldW4cWONHTtWFy9evO4xsrKylJGRYfcBAAAAgKJW7K+w/V5eXp6GDx+uBx54QI0bN7a1P/nkk6pZs6ZCQ0O1Z88ejRkzRgcOHNDSpUsLPE58fLwmTJhQVGUDAAAAQIFKVGAbMmSI9u3bp40bN9q1Dx482PbrJk2aqGrVqurYsaOOHDmi2rVr5zvO2LFjNXLkSNt2RkaGwsLCnFc4AAAAABSgxAS2oUOHasWKFdqwYYOqV69+w74RERGSpMOHDxcY2Ly9veXt7e2UOgEAAADgVhX7wGaM0fPPP69ly5Zp3bp1Cg8Pv+l3du3aJUmqWrWqk6sDboxXNgAAAOBGin1gGzJkiD7++GN98cUXqlChglJSUiRJ/v7+8vHx0ZEjR/Txxx+ra9euCgwM1J49ezRixAi1a9dOTZs2dXH1AAAAAHB9xT6wzZkzR9KVl2P/XkJCggYMGCCr1apVq1ZpxowZunDhgsLCwtS7d2/99a9/dUG1AAAAAHDrin1gM8bccH9YWJjWr19fRNUAAAAAgOMU+8AGlCYFPfM2olM9F1QCAACAokBgA0oZQh8AAEDx4eHqAgAAAAAABSOwAQAAAICbIrABAAAAgJsisAEAAACAmyKwAQAAAICbIrABAAAAgJsisAEAAACAmyKwAQAAAICb4sXZbqSgFxoDAAAAKL24wgYAAAAAboorbEAxd70rsyM61bvjY1zP7RwbAAAAhUdgA0oobrEFAAAo/rglEgAAAADcFFfYADiEI27NBAAAgD2usAEAAACAm+IKm4vwfBGKM2fNX67SAQAA2OMKGwAAAAC4KQIbAAAAALipUnVL5Lvvvqtp06YpJSVFzZo10zvvvKPWrVu7uiygRHP32xwLqs9dagMAACg1ge3TTz/VyJEjNXfuXEVERGjGjBmKjo7WgQMHFBQU5OryANzA7YQ+ng8FAAAlSakJbNOnT9egQYM0cOBASdLcuXP11Vdfad68eXr55Zft+mZlZSkrK8u2nZ6eLknKyMhwWD2XLmQ67FhAcVTQn6fb/XPhiGMUJH75DwW2D3moToHt7645fEd9r9f/dvrebh0AAKBwrv77wxhTJONZTFGN5EKXL1+Wr6+vPv/8c/Xo0cPWHhsbq7S0NH3xxRd2/cePH68JEyYUcZUAAAAAiosjR47orrvucvo4peIK22+//abc3FwFBwfbtQcHB+unn37K13/s2LEaOXKkbTstLU01a9ZUUlKS/P39nV4vSq+MjAyFhYXpl19+kZ+fn6vLQQnGXENRYa6hqDDXUFTS09NVo0YNVapUqUjGKxWB7XZ5e3vL29s7X7u/vz9/AaBI+Pn5MddQJJhrKCrMNRQV5hqKiodH0Sy4XyqW9a9cubI8PT116tQpu/ZTp04pJCTERVUBAAAAwI2VisBmtVrVsmVLrV692taWl5en1atXKzIy0oWVAQAAAMD1lZpbIkeOHKnY2Fi1atVKrVu31owZM3ThwgXbqpE34u3trddee63A2yQBR2Kuoagw11BUmGsoKsw1FJWinmulYpXIq2bPnm17cXbz5s01a9YsRUREuLosAAAAAChQqQpsAAAAAFCclIpn2AAAAACgOCKwAQAAAICbIrABAAAAgJsisAEAAACAmyKw3YJ3331XtWrVUtmyZRUREaGtW7e6uiQUI/Hx8br33ntVoUIFBQUFqUePHjpw4IBdn0uXLmnIkCEKDAxU+fLl1bt373wvek9KSlK3bt3k6+uroKAgvfjii8rJySnKU0ExM3nyZFksFg0fPtzWxlyDo/z666/q16+fAgMD5ePjoyZNmmj79u22/cYYjRs3TlWrVpWPj4+ioqJ06NAhu2OkpqYqJiZGfn5+CggIUFxcnDIzM4v6VODGcnNz9eqrryo8PFw+Pj6qXbu2Jk2apN+vmcdcQ2Fs2LBB3bt3V2hoqCwWi5YvX26331Hzas+ePWrbtq3Kli2rsLAwTZ069faLNbihxYsXG6vVaubNm2f2799vBg0aZAICAsypU6dcXRqKiejoaJOQkGD27dtndu3aZbp27Wpq1KhhMjMzbX2effZZExYWZlavXm22b99u7rvvPnP//ffb9ufk5JjGjRubqKgos3PnTvP111+bypUrm7Fjx7rilFAMbN261dSqVcs0bdrUDBs2zNbOXIMjpKammpo1a5oBAwaYLVu2mJ9//tl8++235vDhw7Y+kydPNv7+/mb58uVm9+7d5pFHHjHh4eHmf//7n61P586dTbNmzcz3339v/v3vf5s6deqYvn37uuKU4KZef/11ExgYaFasWGGOHj1qlixZYsqXL29mzpxp68NcQ2F8/fXX5pVXXjFLly41ksyyZcvs9jtiXqWnp5vg4GATExNj9u3bZz755BPj4+Nj3n///duqlcB2E61btzZDhgyxbefm5prQ0FATHx/vwqpQnJ0+fdpIMuvXrzfGGJOWlmbKlCljlixZYuvz448/Gklm8+bNxpgrf6l4eHiYlJQUW585c+YYPz8/k5WVVbQnALd3/vx5U7duXZOYmGjat29vC2zMNTjKmDFjTJs2ba67Py8vz4SEhJhp06bZ2tLS0oy3t7f55JNPjDHG/Pe//zWSzLZt22x9vvnmG2OxWMyvv/7qvOJRrHTr1s08/fTTdm29evUyMTExxhjmGhzj2sDmqHn13nvvmYoVK9r993PMmDGmfv36t1Uft0TewOXLl7Vjxw5FRUXZ2jw8PBQVFaXNmze7sDIUZ+np6ZKkSpUqSZJ27Nih7Oxsu3nWoEED1ahRwzbPNm/erCZNmig4ONjWJzo6WhkZGdq/f38RVo/iYMiQIerWrZvdnJKYa3CcL7/8Uq1atdLjjz+uoKAgtWjRQn//+99t+48ePaqUlBS7uebv76+IiAi7uRYQEKBWrVrZ+kRFRcnDw0NbtmwpupOBW7v//vu1evVqHTx4UJK0e/dubdy4UV26dJHEXINzOGpebd68We3atZPVarX1iY6O1oEDB3Tu3LlbrsfrTk+oJPvtt9+Um5tr9w8XSQoODtZPP/3koqpQnOXl5Wn48OF64IEH1LhxY0lSSkqKrFarAgIC7PoGBwcrJSXF1qegeXh1H3DV4sWL9cMPP2jbtm359jHX4Cg///yz5syZo5EjR+ovf/mLtm3bphdeeEFWq1WxsbG2uVLQXPr9XAsKCrLb7+XlpUqVKjHXYPPyyy8rIyNDDRo0kKenp3Jzc/X6668rJiZGkphrcApHzauUlBSFh4fnO8bVfRUrVryleghsQBEaMmSI9u3bp40bN7q6FJRAv/zyi4YNG6bExESVLVvW1eWgBMvLy1OrVq30xhtvSJJatGihffv2ae7cuYqNjXVxdShJPvvsMy1atEgff/yx7r77bu3atUvDhw9XaGgocw2lBrdE3kDlypXl6emZbwW1U6dOKSQkxEVVobgaOnSoVqxYobVr16p69eq29pCQEF2+fFlpaWl2/X8/z0JCQgqch1f3AdKVWx5Pnz6te+65R15eXvLy8tL69es1a9YseXl5KTg4mLkGh6hataoaNWpk19awYUMlJSVJ+n9z5Ub//QwJCdHp06ft9ufk5Cg1NZW5BpsXX3xRL7/8svr06aMmTZroqaee0ogRIxQfHy+JuQbncNS8ctR/UwlsN2C1WtWyZUutXr3a1paXl6fVq1crMjLShZWhODHGaOjQoVq2bJnWrFmT79J4y5YtVaZMGbt5duDAASUlJdnmWWRkpPbu3Wv3F0NiYqL8/Pzy/aMJpVfHjh21d+9e7dq1y/Zp1aqVYmJibL9mrsERHnjggXyvJzl48KBq1qwpSQoPD1dISIjdXMvIyNCWLVvs5lpaWpp27Nhh67NmzRrl5eUpIiKiCM4CxcHFixfl4WH/z1VPT0/l5eVJYq7BORw1ryIjI7VhwwZlZ2fb+iQmJqp+/fq3fDukJJb1v5nFixcbb29vM3/+fPPf//7XDB482AQEBNitoAbcyHPPPWf8/f3NunXrTHJysu1z8eJFW59nn33W1KhRw6xZs8Zs377dREZGmsjISNv+q0utP/zww2bXrl1m5cqVpkqVKiy1jpv6/SqRxjDX4Bhbt241Xl5e5vXXXzeHDh0yixYtMr6+vuajjz6y9Zk8ebIJCAgwX3zxhdmzZ4959NFHC1wSu0WLFmbLli1m48aNpm7duiy1DjuxsbGmWrVqtmX9ly5daipXrmxeeuklWx/mGgrj/PnzZufOnWbnzp1Gkpk+fbrZuXOnOX78uDHGMfMqLS3NBAcHm6eeesrs27fPLF682Pj6+rKsvzO88847pkaNGsZqtZrWrVub77//3tUloRiRVOAnISHB1ud///uf+fOf/2wqVqxofH19Tc+ePU1ycrLdcY4dO2a6dOlifHx8TOXKlc2oUaNMdnZ2EZ8NiptrAxtzDY7yr3/9yzRu3Nh4e3ubBg0amA8++MBuf15ennn11VdNcHCw8fb2Nh07djQHDhyw63P27FnTt29fU758eePn52cGDhxozp8/X5SnATeXkZFhhg0bZmrUqGHKli1r7rrrLvPKK6/YLZPOXENhrF27tsB/n8XGxhpjHDevdu/ebdq0aWO8vb1NtWrVzOTJk2+7Vosxv3tVPAAAAADAbfAMGwAAAAC4KQIbAAAAALgpAhsAAAAAuCkCGwAAAAC4KQIbAAAAALgpAhsAAAAAuCkCGwAAAAC4KQIbAAAAALgpAhsAAAAAuCkCGwAAAAC4KQIbAAAAALip/w/9zCOEDE9MigAAAABJRU5ErkJggg==",
            "text/plain": [
              "<Figure size 1000x600 with 2 Axes>"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        }
      ],
      "source": [
        "dataset_clean[\"length of text\"] = dataset_clean[\"text\"].map(len)\n",
        "ax = dataset_clean.plot.hist(column=[\"length of text\"], by=\"_label_\", bins=50, alpha=0.5, figsize=(10, 6), title=\"Distribution of string length per class\", xlim=[0, 1000])"
      ]
    },
    {
      "attachments": {},
      "cell_type": "markdown",
      "metadata": {
        "id": "HgDwJos4FgJK"
      },
      "source": [
        "## Feature engineering"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "vX9IFcHYFi7f",
        "outputId": "b689dd2d-1e25-42d6-8987-61d6f5d08b1a"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "Feature Engineering method: Binary (one hot encoding)\n"
          ]
        }
      ],
      "source": [
        "def feat_eng_text_df(in_df, text_col, labels_col, config_dict):\n",
        "  if \"CountVectorizer-binary\" == config_dict[\"feature_eng_details\"]:\n",
        "    print(\"Feature Engineering method: Binary (one hot encoding)\")\n",
        "    countvectorizer = CountVectorizer(ngram_range=(config_dict[\"ngram_range_min\"], config_dict[\"ngram_range_max\"]),\n",
        "                                      stop_words='english',\n",
        "                                      max_features=config_dict[\"max_features\"],\n",
        "                                      binary=True)\n",
        "\n",
        "  elif \"CountVectorizer-BOW\" == config_dict[\"feature_eng_details\"]:\n",
        "    print(\"Feature Engineering method: Bag of words\")\n",
        "    countvectorizer = CountVectorizer(ngram_range=(config_dict[\"ngram_range_min\"], config_dict[\"ngram_range_max\"]),\n",
        "                                      stop_words='english',\n",
        "                                      max_features=config_dict[\"max_features\"],\n",
        "                                      binary=False)\n",
        "\n",
        "  out_arr = countvectorizer.fit_transform(in_df[text_col])\n",
        "  count_tokens = countvectorizer.get_feature_names_out()\n",
        "  out_df = pd.DataFrame(data = out_arr.toarray(),columns = count_tokens)\n",
        "  out_df[labels_col] = list(in_df[labels_col])\n",
        "  return out_df\n",
        "\n",
        "\n",
        "if config_dict[\"do_feature_eng\"]:\n",
        "  dataset_feat_eng = feat_eng_text_df(dataset_clean, 'text', '_label_', config_dict)\n",
        "else:\n",
        "  # This option isn't being supported, the notebook would fail. This option is\n",
        "  # here to cater for a ML pipeline that uses deep learning language models that consume text, and not engineered features.\n",
        "  dataset_feat_eng = dataset_clean.copy()"
      ]
    },
    {
      "attachments": {},
      "cell_type": "markdown",
      "metadata": {
        "id": "f_WMzIFM_i6A"
      },
      "source": [
        "## Exploring the new numerical features  "
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 255
        },
        "id": "Aqz7oNbE_328",
        "outputId": "1dce6b50-b91f-4e4a-be5b-dc9948cd95c4"
      },
      "outputs": [
        {
          "data": {
            "application/vnd.google.colaboratory.intrinsic+json": {
              "type": "dataframe",
              "variable_name": "dataset_feat_eng"
            },
            "text/html": [
              "\n",
              "  <div id=\"df-0d0e3e1f-434f-4b1e-b1cd-f9fb51e2263b\" class=\"colab-df-container\">\n",
              "    <div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>abdominal</th>\n",
              "      <th>ability</th>\n",
              "      <th>able</th>\n",
              "      <th>abnormal</th>\n",
              "      <th>abnormalities</th>\n",
              "      <th>absence</th>\n",
              "      <th>acceptable</th>\n",
              "      <th>access</th>\n",
              "      <th>according</th>\n",
              "      <th>account</th>\n",
              "      <th>...</th>\n",
              "      <th>wide</th>\n",
              "      <th>women</th>\n",
              "      <th>work</th>\n",
              "      <th>wound</th>\n",
              "      <th>year</th>\n",
              "      <th>years</th>\n",
              "      <th>young</th>\n",
              "      <th>younger</th>\n",
              "      <th>zeroone</th>\n",
              "      <th>_label_</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>...</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>...</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>...</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3</th>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>...</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>4</th>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>...</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "<p>5 rows × 1001 columns</p>\n",
              "</div>\n",
              "    <div class=\"colab-df-buttons\">\n",
              "\n",
              "  <div class=\"colab-df-container\">\n",
              "    <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-0d0e3e1f-434f-4b1e-b1cd-f9fb51e2263b')\"\n",
              "            title=\"Convert this dataframe to an interactive table.\"\n",
              "            style=\"display:none;\">\n",
              "\n",
              "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\" viewBox=\"0 -960 960 960\">\n",
              "    <path d=\"M120-120v-720h720v720H120Zm60-500h600v-160H180v160Zm220 220h160v-160H400v160Zm0 220h160v-160H400v160ZM180-400h160v-160H180v160Zm440 0h160v-160H620v160ZM180-180h160v-160H180v160Zm440 0h160v-160H620v160Z\"/>\n",
              "  </svg>\n",
              "    </button>\n",
              "\n",
              "  <style>\n",
              "    .colab-df-container {\n",
              "      display:flex;\n",
              "      gap: 12px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert {\n",
              "      background-color: #E8F0FE;\n",
              "      border: none;\n",
              "      border-radius: 50%;\n",
              "      cursor: pointer;\n",
              "      display: none;\n",
              "      fill: #1967D2;\n",
              "      height: 32px;\n",
              "      padding: 0 0 0 0;\n",
              "      width: 32px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert:hover {\n",
              "      background-color: #E2EBFA;\n",
              "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "      fill: #174EA6;\n",
              "    }\n",
              "\n",
              "    .colab-df-buttons div {\n",
              "      margin-bottom: 4px;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert {\n",
              "      background-color: #3B4455;\n",
              "      fill: #D2E3FC;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert:hover {\n",
              "      background-color: #434B5C;\n",
              "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
              "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
              "      fill: #FFFFFF;\n",
              "    }\n",
              "  </style>\n",
              "\n",
              "    <script>\n",
              "      const buttonEl =\n",
              "        document.querySelector('#df-0d0e3e1f-434f-4b1e-b1cd-f9fb51e2263b button.colab-df-convert');\n",
              "      buttonEl.style.display =\n",
              "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "\n",
              "      async function convertToInteractive(key) {\n",
              "        const element = document.querySelector('#df-0d0e3e1f-434f-4b1e-b1cd-f9fb51e2263b');\n",
              "        const dataTable =\n",
              "          await google.colab.kernel.invokeFunction('convertToInteractive',\n",
              "                                                    [key], {});\n",
              "        if (!dataTable) return;\n",
              "\n",
              "        const docLinkHtml = 'Like what you see? Visit the ' +\n",
              "          '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
              "          + ' to learn more about interactive tables.';\n",
              "        element.innerHTML = '';\n",
              "        dataTable['output_type'] = 'display_data';\n",
              "        await google.colab.output.renderOutput(dataTable, element);\n",
              "        const docLink = document.createElement('div');\n",
              "        docLink.innerHTML = docLinkHtml;\n",
              "        element.appendChild(docLink);\n",
              "      }\n",
              "    </script>\n",
              "  </div>\n",
              "\n",
              "\n",
              "<div id=\"df-29a597c6-5bd5-4bb9-9856-27b8f127c4e6\">\n",
              "  <button class=\"colab-df-quickchart\" onclick=\"quickchart('df-29a597c6-5bd5-4bb9-9856-27b8f127c4e6')\"\n",
              "            title=\"Suggest charts\"\n",
              "            style=\"display:none;\">\n",
              "\n",
              "<svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
              "     width=\"24px\">\n",
              "    <g>\n",
              "        <path d=\"M19 3H5c-1.1 0-2 .9-2 2v14c0 1.1.9 2 2 2h14c1.1 0 2-.9 2-2V5c0-1.1-.9-2-2-2zM9 17H7v-7h2v7zm4 0h-2V7h2v10zm4 0h-2v-4h2v4z\"/>\n",
              "    </g>\n",
              "</svg>\n",
              "  </button>\n",
              "\n",
              "<style>\n",
              "  .colab-df-quickchart {\n",
              "      --bg-color: #E8F0FE;\n",
              "      --fill-color: #1967D2;\n",
              "      --hover-bg-color: #E2EBFA;\n",
              "      --hover-fill-color: #174EA6;\n",
              "      --disabled-fill-color: #AAA;\n",
              "      --disabled-bg-color: #DDD;\n",
              "  }\n",
              "\n",
              "  [theme=dark] .colab-df-quickchart {\n",
              "      --bg-color: #3B4455;\n",
              "      --fill-color: #D2E3FC;\n",
              "      --hover-bg-color: #434B5C;\n",
              "      --hover-fill-color: #FFFFFF;\n",
              "      --disabled-bg-color: #3B4455;\n",
              "      --disabled-fill-color: #666;\n",
              "  }\n",
              "\n",
              "  .colab-df-quickchart {\n",
              "    background-color: var(--bg-color);\n",
              "    border: none;\n",
              "    border-radius: 50%;\n",
              "    cursor: pointer;\n",
              "    display: none;\n",
              "    fill: var(--fill-color);\n",
              "    height: 32px;\n",
              "    padding: 0;\n",
              "    width: 32px;\n",
              "  }\n",
              "\n",
              "  .colab-df-quickchart:hover {\n",
              "    background-color: var(--hover-bg-color);\n",
              "    box-shadow: 0 1px 2px rgba(60, 64, 67, 0.3), 0 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "    fill: var(--button-hover-fill-color);\n",
              "  }\n",
              "\n",
              "  .colab-df-quickchart-complete:disabled,\n",
              "  .colab-df-quickchart-complete:disabled:hover {\n",
              "    background-color: var(--disabled-bg-color);\n",
              "    fill: var(--disabled-fill-color);\n",
              "    box-shadow: none;\n",
              "  }\n",
              "\n",
              "  .colab-df-spinner {\n",
              "    border: 2px solid var(--fill-color);\n",
              "    border-color: transparent;\n",
              "    border-bottom-color: var(--fill-color);\n",
              "    animation:\n",
              "      spin 1s steps(1) infinite;\n",
              "  }\n",
              "\n",
              "  @keyframes spin {\n",
              "    0% {\n",
              "      border-color: transparent;\n",
              "      border-bottom-color: var(--fill-color);\n",
              "      border-left-color: var(--fill-color);\n",
              "    }\n",
              "    20% {\n",
              "      border-color: transparent;\n",
              "      border-left-color: var(--fill-color);\n",
              "      border-top-color: var(--fill-color);\n",
              "    }\n",
              "    30% {\n",
              "      border-color: transparent;\n",
              "      border-left-color: var(--fill-color);\n",
              "      border-top-color: var(--fill-color);\n",
              "      border-right-color: var(--fill-color);\n",
              "    }\n",
              "    40% {\n",
              "      border-color: transparent;\n",
              "      border-right-color: var(--fill-color);\n",
              "      border-top-color: var(--fill-color);\n",
              "    }\n",
              "    60% {\n",
              "      border-color: transparent;\n",
              "      border-right-color: var(--fill-color);\n",
              "    }\n",
              "    80% {\n",
              "      border-color: transparent;\n",
              "      border-right-color: var(--fill-color);\n",
              "      border-bottom-color: var(--fill-color);\n",
              "    }\n",
              "    90% {\n",
              "      border-color: transparent;\n",
              "      border-bottom-color: var(--fill-color);\n",
              "    }\n",
              "  }\n",
              "</style>\n",
              "\n",
              "  <script>\n",
              "    async function quickchart(key) {\n",
              "      const quickchartButtonEl =\n",
              "        document.querySelector('#' + key + ' button');\n",
              "      quickchartButtonEl.disabled = true;  // To prevent multiple clicks.\n",
              "      quickchartButtonEl.classList.add('colab-df-spinner');\n",
              "      try {\n",
              "        const charts = await google.colab.kernel.invokeFunction(\n",
              "            'suggestCharts', [key], {});\n",
              "      } catch (error) {\n",
              "        console.error('Error during call to suggestCharts:', error);\n",
              "      }\n",
              "      quickchartButtonEl.classList.remove('colab-df-spinner');\n",
              "      quickchartButtonEl.classList.add('colab-df-quickchart-complete');\n",
              "    }\n",
              "    (() => {\n",
              "      let quickchartButtonEl =\n",
              "        document.querySelector('#df-29a597c6-5bd5-4bb9-9856-27b8f127c4e6 button');\n",
              "      quickchartButtonEl.style.display =\n",
              "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "    })();\n",
              "  </script>\n",
              "</div>\n",
              "\n",
              "    </div>\n",
              "  </div>\n"
            ],
            "text/plain": [
              "   abdominal  ability  able  abnormal  abnormalities  absence  acceptable  \\\n",
              "0          0        0     0         0              0        0           0   \n",
              "1          0        0     0         0              0        0           0   \n",
              "2          0        0     0         0              0        0           0   \n",
              "3          0        0     0         0              0        0           0   \n",
              "4          0        0     0         0              0        0           0   \n",
              "\n",
              "   access  according  account  ...  wide  women  work  wound  year  years  \\\n",
              "0       0          0        0  ...     0      0     0      0     0      0   \n",
              "1       0          0        0  ...     0      0     0      0     0      0   \n",
              "2       0          0        0  ...     0      0     0      0     0      0   \n",
              "3       0          0        0  ...     0      0     0      0     0      0   \n",
              "4       0          0        0  ...     0      0     0      0     0      0   \n",
              "\n",
              "   young  younger  zeroone  _label_  \n",
              "0      0        0        0        0  \n",
              "1      0        0        0        0  \n",
              "2      0        0        0        0  \n",
              "3      0        0        0        0  \n",
              "4      0        0        0        0  \n",
              "\n",
              "[5 rows x 1001 columns]"
            ]
          },
          "execution_count": 16,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "dataset_feat_eng.head()"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 193
        },
        "id": "8ZPYzjltUZ3Q",
        "outputId": "ad29ab05-ad66-4eec-a5a4-fb20daee2ab4"
      },
      "outputs": [
        {
          "data": {
            "application/vnd.google.colaboratory.intrinsic+json": {
              "type": "dataframe"
            },
            "text/html": [
              "\n",
              "  <div id=\"df-4e8896ef-d82c-4615-8e22-3d89e149daa6\" class=\"colab-df-container\">\n",
              "    <div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>abdominal</th>\n",
              "      <th>ability</th>\n",
              "      <th>able</th>\n",
              "      <th>abnormal</th>\n",
              "      <th>abnormalities</th>\n",
              "      <th>absence</th>\n",
              "      <th>acceptable</th>\n",
              "      <th>access</th>\n",
              "      <th>according</th>\n",
              "      <th>account</th>\n",
              "      <th>...</th>\n",
              "      <th>wide</th>\n",
              "      <th>women</th>\n",
              "      <th>work</th>\n",
              "      <th>wound</th>\n",
              "      <th>year</th>\n",
              "      <th>years</th>\n",
              "      <th>young</th>\n",
              "      <th>younger</th>\n",
              "      <th>zeroone</th>\n",
              "      <th>_label_</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>min</th>\n",
              "      <td>0.00000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>...</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.000000</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>max</th>\n",
              "      <td>1.00000</td>\n",
              "      <td>1.000000</td>\n",
              "      <td>1.000000</td>\n",
              "      <td>1.000000</td>\n",
              "      <td>1.000000</td>\n",
              "      <td>1.000000</td>\n",
              "      <td>1.000000</td>\n",
              "      <td>1.000000</td>\n",
              "      <td>1.000000</td>\n",
              "      <td>1.000000</td>\n",
              "      <td>...</td>\n",
              "      <td>1.000000</td>\n",
              "      <td>1.000000</td>\n",
              "      <td>1.000000</td>\n",
              "      <td>1.000000</td>\n",
              "      <td>1.000000</td>\n",
              "      <td>1.000000</td>\n",
              "      <td>1.000000</td>\n",
              "      <td>1.000000</td>\n",
              "      <td>1.000000</td>\n",
              "      <td>1.000000</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>mean</th>\n",
              "      <td>0.00438</td>\n",
              "      <td>0.004956</td>\n",
              "      <td>0.003688</td>\n",
              "      <td>0.003458</td>\n",
              "      <td>0.003804</td>\n",
              "      <td>0.003804</td>\n",
              "      <td>0.003804</td>\n",
              "      <td>0.004034</td>\n",
              "      <td>0.004841</td>\n",
              "      <td>0.006339</td>\n",
              "      <td>...</td>\n",
              "      <td>0.002766</td>\n",
              "      <td>0.026971</td>\n",
              "      <td>0.006224</td>\n",
              "      <td>0.002651</td>\n",
              "      <td>0.004034</td>\n",
              "      <td>0.017865</td>\n",
              "      <td>0.007492</td>\n",
              "      <td>0.003804</td>\n",
              "      <td>0.002997</td>\n",
              "      <td>0.253458</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "<p>3 rows × 1001 columns</p>\n",
              "</div>\n",
              "    <div class=\"colab-df-buttons\">\n",
              "\n",
              "  <div class=\"colab-df-container\">\n",
              "    <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-4e8896ef-d82c-4615-8e22-3d89e149daa6')\"\n",
              "            title=\"Convert this dataframe to an interactive table.\"\n",
              "            style=\"display:none;\">\n",
              "\n",
              "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\" viewBox=\"0 -960 960 960\">\n",
              "    <path d=\"M120-120v-720h720v720H120Zm60-500h600v-160H180v160Zm220 220h160v-160H400v160Zm0 220h160v-160H400v160ZM180-400h160v-160H180v160Zm440 0h160v-160H620v160ZM180-180h160v-160H180v160Zm440 0h160v-160H620v160Z\"/>\n",
              "  </svg>\n",
              "    </button>\n",
              "\n",
              "  <style>\n",
              "    .colab-df-container {\n",
              "      display:flex;\n",
              "      gap: 12px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert {\n",
              "      background-color: #E8F0FE;\n",
              "      border: none;\n",
              "      border-radius: 50%;\n",
              "      cursor: pointer;\n",
              "      display: none;\n",
              "      fill: #1967D2;\n",
              "      height: 32px;\n",
              "      padding: 0 0 0 0;\n",
              "      width: 32px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert:hover {\n",
              "      background-color: #E2EBFA;\n",
              "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "      fill: #174EA6;\n",
              "    }\n",
              "\n",
              "    .colab-df-buttons div {\n",
              "      margin-bottom: 4px;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert {\n",
              "      background-color: #3B4455;\n",
              "      fill: #D2E3FC;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert:hover {\n",
              "      background-color: #434B5C;\n",
              "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
              "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
              "      fill: #FFFFFF;\n",
              "    }\n",
              "  </style>\n",
              "\n",
              "    <script>\n",
              "      const buttonEl =\n",
              "        document.querySelector('#df-4e8896ef-d82c-4615-8e22-3d89e149daa6 button.colab-df-convert');\n",
              "      buttonEl.style.display =\n",
              "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "\n",
              "      async function convertToInteractive(key) {\n",
              "        const element = document.querySelector('#df-4e8896ef-d82c-4615-8e22-3d89e149daa6');\n",
              "        const dataTable =\n",
              "          await google.colab.kernel.invokeFunction('convertToInteractive',\n",
              "                                                    [key], {});\n",
              "        if (!dataTable) return;\n",
              "\n",
              "        const docLinkHtml = 'Like what you see? Visit the ' +\n",
              "          '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
              "          + ' to learn more about interactive tables.';\n",
              "        element.innerHTML = '';\n",
              "        dataTable['output_type'] = 'display_data';\n",
              "        await google.colab.output.renderOutput(dataTable, element);\n",
              "        const docLink = document.createElement('div');\n",
              "        docLink.innerHTML = docLinkHtml;\n",
              "        element.appendChild(docLink);\n",
              "      }\n",
              "    </script>\n",
              "  </div>\n",
              "\n",
              "\n",
              "<div id=\"df-f440949a-9f33-4fd8-af31-1b79d819d29f\">\n",
              "  <button class=\"colab-df-quickchart\" onclick=\"quickchart('df-f440949a-9f33-4fd8-af31-1b79d819d29f')\"\n",
              "            title=\"Suggest charts\"\n",
              "            style=\"display:none;\">\n",
              "\n",
              "<svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
              "     width=\"24px\">\n",
              "    <g>\n",
              "        <path d=\"M19 3H5c-1.1 0-2 .9-2 2v14c0 1.1.9 2 2 2h14c1.1 0 2-.9 2-2V5c0-1.1-.9-2-2-2zM9 17H7v-7h2v7zm4 0h-2V7h2v10zm4 0h-2v-4h2v4z\"/>\n",
              "    </g>\n",
              "</svg>\n",
              "  </button>\n",
              "\n",
              "<style>\n",
              "  .colab-df-quickchart {\n",
              "      --bg-color: #E8F0FE;\n",
              "      --fill-color: #1967D2;\n",
              "      --hover-bg-color: #E2EBFA;\n",
              "      --hover-fill-color: #174EA6;\n",
              "      --disabled-fill-color: #AAA;\n",
              "      --disabled-bg-color: #DDD;\n",
              "  }\n",
              "\n",
              "  [theme=dark] .colab-df-quickchart {\n",
              "      --bg-color: #3B4455;\n",
              "      --fill-color: #D2E3FC;\n",
              "      --hover-bg-color: #434B5C;\n",
              "      --hover-fill-color: #FFFFFF;\n",
              "      --disabled-bg-color: #3B4455;\n",
              "      --disabled-fill-color: #666;\n",
              "  }\n",
              "\n",
              "  .colab-df-quickchart {\n",
              "    background-color: var(--bg-color);\n",
              "    border: none;\n",
              "    border-radius: 50%;\n",
              "    cursor: pointer;\n",
              "    display: none;\n",
              "    fill: var(--fill-color);\n",
              "    height: 32px;\n",
              "    padding: 0;\n",
              "    width: 32px;\n",
              "  }\n",
              "\n",
              "  .colab-df-quickchart:hover {\n",
              "    background-color: var(--hover-bg-color);\n",
              "    box-shadow: 0 1px 2px rgba(60, 64, 67, 0.3), 0 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "    fill: var(--button-hover-fill-color);\n",
              "  }\n",
              "\n",
              "  .colab-df-quickchart-complete:disabled,\n",
              "  .colab-df-quickchart-complete:disabled:hover {\n",
              "    background-color: var(--disabled-bg-color);\n",
              "    fill: var(--disabled-fill-color);\n",
              "    box-shadow: none;\n",
              "  }\n",
              "\n",
              "  .colab-df-spinner {\n",
              "    border: 2px solid var(--fill-color);\n",
              "    border-color: transparent;\n",
              "    border-bottom-color: var(--fill-color);\n",
              "    animation:\n",
              "      spin 1s steps(1) infinite;\n",
              "  }\n",
              "\n",
              "  @keyframes spin {\n",
              "    0% {\n",
              "      border-color: transparent;\n",
              "      border-bottom-color: var(--fill-color);\n",
              "      border-left-color: var(--fill-color);\n",
              "    }\n",
              "    20% {\n",
              "      border-color: transparent;\n",
              "      border-left-color: var(--fill-color);\n",
              "      border-top-color: var(--fill-color);\n",
              "    }\n",
              "    30% {\n",
              "      border-color: transparent;\n",
              "      border-left-color: var(--fill-color);\n",
              "      border-top-color: var(--fill-color);\n",
              "      border-right-color: var(--fill-color);\n",
              "    }\n",
              "    40% {\n",
              "      border-color: transparent;\n",
              "      border-right-color: var(--fill-color);\n",
              "      border-top-color: var(--fill-color);\n",
              "    }\n",
              "    60% {\n",
              "      border-color: transparent;\n",
              "      border-right-color: var(--fill-color);\n",
              "    }\n",
              "    80% {\n",
              "      border-color: transparent;\n",
              "      border-right-color: var(--fill-color);\n",
              "      border-bottom-color: var(--fill-color);\n",
              "    }\n",
              "    90% {\n",
              "      border-color: transparent;\n",
              "      border-bottom-color: var(--fill-color);\n",
              "    }\n",
              "  }\n",
              "</style>\n",
              "\n",
              "  <script>\n",
              "    async function quickchart(key) {\n",
              "      const quickchartButtonEl =\n",
              "        document.querySelector('#' + key + ' button');\n",
              "      quickchartButtonEl.disabled = true;  // To prevent multiple clicks.\n",
              "      quickchartButtonEl.classList.add('colab-df-spinner');\n",
              "      try {\n",
              "        const charts = await google.colab.kernel.invokeFunction(\n",
              "            'suggestCharts', [key], {});\n",
              "      } catch (error) {\n",
              "        console.error('Error during call to suggestCharts:', error);\n",
              "      }\n",
              "      quickchartButtonEl.classList.remove('colab-df-spinner');\n",
              "      quickchartButtonEl.classList.add('colab-df-quickchart-complete');\n",
              "    }\n",
              "    (() => {\n",
              "      let quickchartButtonEl =\n",
              "        document.querySelector('#df-f440949a-9f33-4fd8-af31-1b79d819d29f button');\n",
              "      quickchartButtonEl.style.display =\n",
              "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "    })();\n",
              "  </script>\n",
              "</div>\n",
              "\n",
              "    </div>\n",
              "  </div>\n"
            ],
            "text/plain": [
              "      abdominal   ability      able  abnormal  abnormalities   absence  \\\n",
              "min     0.00000  0.000000  0.000000  0.000000       0.000000  0.000000   \n",
              "max     1.00000  1.000000  1.000000  1.000000       1.000000  1.000000   \n",
              "mean    0.00438  0.004956  0.003688  0.003458       0.003804  0.003804   \n",
              "\n",
              "      acceptable    access  according   account  ...      wide     women  \\\n",
              "min     0.000000  0.000000   0.000000  0.000000  ...  0.000000  0.000000   \n",
              "max     1.000000  1.000000   1.000000  1.000000  ...  1.000000  1.000000   \n",
              "mean    0.003804  0.004034   0.004841  0.006339  ...  0.002766  0.026971   \n",
              "\n",
              "          work     wound      year     years     young   younger   zeroone  \\\n",
              "min   0.000000  0.000000  0.000000  0.000000  0.000000  0.000000  0.000000   \n",
              "max   1.000000  1.000000  1.000000  1.000000  1.000000  1.000000  1.000000   \n",
              "mean  0.006224  0.002651  0.004034  0.017865  0.007492  0.003804  0.002997   \n",
              "\n",
              "       _label_  \n",
              "min   0.000000  \n",
              "max   1.000000  \n",
              "mean  0.253458  \n",
              "\n",
              "[3 rows x 1001 columns]"
            ]
          },
          "execution_count": 17,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "dataset_feat_eng.describe().loc[['min', 'max', 'mean']]"
      ]
    },
    {
      "attachments": {},
      "cell_type": "markdown",
      "metadata": {
        "id": "Vf-JsLSKC-Vk"
      },
      "source": [
        "## Split to Train/Test"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "RQzV5OeY6RaZ"
      },
      "outputs": [],
      "source": [
        "dataset_feat_eng_test = dataset_feat_eng.sample(frac=config_dict[\"test_size\"],random_state=config_dict['seed'])\n",
        "dataset_feat_eng_train = dataset_feat_eng.drop(dataset_feat_eng_test.index)"
      ]
    },
    {
      "attachments": {},
      "cell_type": "markdown",
      "metadata": {
        "id": "TfsphIcZFZEn"
      },
      "source": [
        "## Preliminary statistical analysis and feasibility study\n",
        "This process is perhaps the most valuable for the preliminary study prior to applying ML.  \n",
        "This is where we measure the relationship between \"X\" and \"Y\" so to see whether there is a \"correlation\".  \n",
        "\n",
        "If this were a regression problem, where X and Y are numerical, then it would make sense to evaluate the correlation between X and Y, so to learn whether one could expect a linear regression model to yield good results.\n",
        "\n",
        "Since neither X nor Y are numerical in their nature, we seek to evaluated the **statistical dependence** between them, so to know whether a model would have any \"signal\" to pick up on.  "
      ]
    },
    {
      "attachments": {},
      "cell_type": "markdown",
      "metadata": {
        "id": "k-ZtgfrJ9nhl"
      },
      "source": [
        "Calc:  \n",
        "**P(feature | class)**"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 206
        },
        "id": "HPvdzu9utqMC",
        "outputId": "6f9322d6-61e3-42ae-9bb1-70cab7f5e499"
      },
      "outputs": [
        {
          "data": {
            "application/vnd.google.colaboratory.intrinsic+json": {
              "summary": "{\n  \"name\": \"means_by_class\",\n  \"rows\": 1000,\n  \"fields\": [\n    {\n      \"column\": 0,\n      \"properties\": {\n        \"dtype\": \"number\",\n        \"std\": 0.009844111122505734,\n        \"min\": 0.0,\n        \"max\": 0.15294117647058825,\n        \"num_unique_values\": 135,\n        \"samples\": [\n          0.01197110423116615,\n          0.017956656346749224,\n          0.02518059855521156\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": 1,\n      \"properties\": {\n        \"dtype\": \"number\",\n        \"std\": 0.012863403759899528,\n        \"min\": 0.0,\n        \"max\": 0.20938628158844766,\n        \"num_unique_values\": 74,\n        \"samples\": [\n          0.006016847172081829,\n          0.048134777376654635,\n          0.0048134777376654635\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    }\n  ]\n}",
              "type": "dataframe",
              "variable_name": "means_by_class"
            },
            "text/html": [
              "\n",
              "  <div id=\"df-ea2fb93b-9847-4b13-9344-d0eb604805e8\" class=\"colab-df-container\">\n",
              "    <div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th>_label_</th>\n",
              "      <th>0</th>\n",
              "      <th>1</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>abdominal</th>\n",
              "      <td>0.004954</td>\n",
              "      <td>0.002407</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>ability</th>\n",
              "      <td>0.004747</td>\n",
              "      <td>0.005415</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>able</th>\n",
              "      <td>0.004954</td>\n",
              "      <td>0.001805</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>abnormal</th>\n",
              "      <td>0.005160</td>\n",
              "      <td>0.000602</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>abnormalities</th>\n",
              "      <td>0.004334</td>\n",
              "      <td>0.001805</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>\n",
              "    <div class=\"colab-df-buttons\">\n",
              "\n",
              "  <div class=\"colab-df-container\">\n",
              "    <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-ea2fb93b-9847-4b13-9344-d0eb604805e8')\"\n",
              "            title=\"Convert this dataframe to an interactive table.\"\n",
              "            style=\"display:none;\">\n",
              "\n",
              "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\" viewBox=\"0 -960 960 960\">\n",
              "    <path d=\"M120-120v-720h720v720H120Zm60-500h600v-160H180v160Zm220 220h160v-160H400v160Zm0 220h160v-160H400v160ZM180-400h160v-160H180v160Zm440 0h160v-160H620v160ZM180-180h160v-160H180v160Zm440 0h160v-160H620v160Z\"/>\n",
              "  </svg>\n",
              "    </button>\n",
              "\n",
              "  <style>\n",
              "    .colab-df-container {\n",
              "      display:flex;\n",
              "      gap: 12px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert {\n",
              "      background-color: #E8F0FE;\n",
              "      border: none;\n",
              "      border-radius: 50%;\n",
              "      cursor: pointer;\n",
              "      display: none;\n",
              "      fill: #1967D2;\n",
              "      height: 32px;\n",
              "      padding: 0 0 0 0;\n",
              "      width: 32px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert:hover {\n",
              "      background-color: #E2EBFA;\n",
              "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "      fill: #174EA6;\n",
              "    }\n",
              "\n",
              "    .colab-df-buttons div {\n",
              "      margin-bottom: 4px;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert {\n",
              "      background-color: #3B4455;\n",
              "      fill: #D2E3FC;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert:hover {\n",
              "      background-color: #434B5C;\n",
              "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
              "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
              "      fill: #FFFFFF;\n",
              "    }\n",
              "  </style>\n",
              "\n",
              "    <script>\n",
              "      const buttonEl =\n",
              "        document.querySelector('#df-ea2fb93b-9847-4b13-9344-d0eb604805e8 button.colab-df-convert');\n",
              "      buttonEl.style.display =\n",
              "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "\n",
              "      async function convertToInteractive(key) {\n",
              "        const element = document.querySelector('#df-ea2fb93b-9847-4b13-9344-d0eb604805e8');\n",
              "        const dataTable =\n",
              "          await google.colab.kernel.invokeFunction('convertToInteractive',\n",
              "                                                    [key], {});\n",
              "        if (!dataTable) return;\n",
              "\n",
              "        const docLinkHtml = 'Like what you see? Visit the ' +\n",
              "          '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
              "          + ' to learn more about interactive tables.';\n",
              "        element.innerHTML = '';\n",
              "        dataTable['output_type'] = 'display_data';\n",
              "        await google.colab.output.renderOutput(dataTable, element);\n",
              "        const docLink = document.createElement('div');\n",
              "        docLink.innerHTML = docLinkHtml;\n",
              "        element.appendChild(docLink);\n",
              "      }\n",
              "    </script>\n",
              "  </div>\n",
              "\n",
              "\n",
              "<div id=\"df-faac2bff-522c-4cf2-bced-adce206f03eb\">\n",
              "  <button class=\"colab-df-quickchart\" onclick=\"quickchart('df-faac2bff-522c-4cf2-bced-adce206f03eb')\"\n",
              "            title=\"Suggest charts\"\n",
              "            style=\"display:none;\">\n",
              "\n",
              "<svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
              "     width=\"24px\">\n",
              "    <g>\n",
              "        <path d=\"M19 3H5c-1.1 0-2 .9-2 2v14c0 1.1.9 2 2 2h14c1.1 0 2-.9 2-2V5c0-1.1-.9-2-2-2zM9 17H7v-7h2v7zm4 0h-2V7h2v10zm4 0h-2v-4h2v4z\"/>\n",
              "    </g>\n",
              "</svg>\n",
              "  </button>\n",
              "\n",
              "<style>\n",
              "  .colab-df-quickchart {\n",
              "      --bg-color: #E8F0FE;\n",
              "      --fill-color: #1967D2;\n",
              "      --hover-bg-color: #E2EBFA;\n",
              "      --hover-fill-color: #174EA6;\n",
              "      --disabled-fill-color: #AAA;\n",
              "      --disabled-bg-color: #DDD;\n",
              "  }\n",
              "\n",
              "  [theme=dark] .colab-df-quickchart {\n",
              "      --bg-color: #3B4455;\n",
              "      --fill-color: #D2E3FC;\n",
              "      --hover-bg-color: #434B5C;\n",
              "      --hover-fill-color: #FFFFFF;\n",
              "      --disabled-bg-color: #3B4455;\n",
              "      --disabled-fill-color: #666;\n",
              "  }\n",
              "\n",
              "  .colab-df-quickchart {\n",
              "    background-color: var(--bg-color);\n",
              "    border: none;\n",
              "    border-radius: 50%;\n",
              "    cursor: pointer;\n",
              "    display: none;\n",
              "    fill: var(--fill-color);\n",
              "    height: 32px;\n",
              "    padding: 0;\n",
              "    width: 32px;\n",
              "  }\n",
              "\n",
              "  .colab-df-quickchart:hover {\n",
              "    background-color: var(--hover-bg-color);\n",
              "    box-shadow: 0 1px 2px rgba(60, 64, 67, 0.3), 0 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "    fill: var(--button-hover-fill-color);\n",
              "  }\n",
              "\n",
              "  .colab-df-quickchart-complete:disabled,\n",
              "  .colab-df-quickchart-complete:disabled:hover {\n",
              "    background-color: var(--disabled-bg-color);\n",
              "    fill: var(--disabled-fill-color);\n",
              "    box-shadow: none;\n",
              "  }\n",
              "\n",
              "  .colab-df-spinner {\n",
              "    border: 2px solid var(--fill-color);\n",
              "    border-color: transparent;\n",
              "    border-bottom-color: var(--fill-color);\n",
              "    animation:\n",
              "      spin 1s steps(1) infinite;\n",
              "  }\n",
              "\n",
              "  @keyframes spin {\n",
              "    0% {\n",
              "      border-color: transparent;\n",
              "      border-bottom-color: var(--fill-color);\n",
              "      border-left-color: var(--fill-color);\n",
              "    }\n",
              "    20% {\n",
              "      border-color: transparent;\n",
              "      border-left-color: var(--fill-color);\n",
              "      border-top-color: var(--fill-color);\n",
              "    }\n",
              "    30% {\n",
              "      border-color: transparent;\n",
              "      border-left-color: var(--fill-color);\n",
              "      border-top-color: var(--fill-color);\n",
              "      border-right-color: var(--fill-color);\n",
              "    }\n",
              "    40% {\n",
              "      border-color: transparent;\n",
              "      border-right-color: var(--fill-color);\n",
              "      border-top-color: var(--fill-color);\n",
              "    }\n",
              "    60% {\n",
              "      border-color: transparent;\n",
              "      border-right-color: var(--fill-color);\n",
              "    }\n",
              "    80% {\n",
              "      border-color: transparent;\n",
              "      border-right-color: var(--fill-color);\n",
              "      border-bottom-color: var(--fill-color);\n",
              "    }\n",
              "    90% {\n",
              "      border-color: transparent;\n",
              "      border-bottom-color: var(--fill-color);\n",
              "    }\n",
              "  }\n",
              "</style>\n",
              "\n",
              "  <script>\n",
              "    async function quickchart(key) {\n",
              "      const quickchartButtonEl =\n",
              "        document.querySelector('#' + key + ' button');\n",
              "      quickchartButtonEl.disabled = true;  // To prevent multiple clicks.\n",
              "      quickchartButtonEl.classList.add('colab-df-spinner');\n",
              "      try {\n",
              "        const charts = await google.colab.kernel.invokeFunction(\n",
              "            'suggestCharts', [key], {});\n",
              "      } catch (error) {\n",
              "        console.error('Error during call to suggestCharts:', error);\n",
              "      }\n",
              "      quickchartButtonEl.classList.remove('colab-df-spinner');\n",
              "      quickchartButtonEl.classList.add('colab-df-quickchart-complete');\n",
              "    }\n",
              "    (() => {\n",
              "      let quickchartButtonEl =\n",
              "        document.querySelector('#df-faac2bff-522c-4cf2-bced-adce206f03eb button');\n",
              "      quickchartButtonEl.style.display =\n",
              "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "    })();\n",
              "  </script>\n",
              "</div>\n",
              "\n",
              "    </div>\n",
              "  </div>\n"
            ],
            "text/plain": [
              "_label_               0         1\n",
              "abdominal      0.004954  0.002407\n",
              "ability        0.004747  0.005415\n",
              "able           0.004954  0.001805\n",
              "abnormal       0.005160  0.000602\n",
              "abnormalities  0.004334  0.001805"
            ]
          },
          "execution_count": 19,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "## Statistics of features per class:\n",
        "means_by_class = dataset_feat_eng_train.groupby(by=[\"_label_\"]).mean().T.sort_index()\n",
        "means_by_class.head()"
      ]
    },
    {
      "attachments": {},
      "cell_type": "markdown",
      "metadata": {
        "id": "LINyT4DBAsf8"
      },
      "source": [
        "Calc the ratio that reflects statistical dependence:  \n",
        "**P(class, feature)/(P(class)P(feature))**  \n",
        "And note that it could be rewritten as:  \n",
        "**P(class | feature)/P(class)**  \n",
        "Or equivalently:  \n",
        "**P(feature | class)/P(feature)**  \n",
        "\n",
        "\\*Note:  \n",
        "The below calculation is assuming that the numerical features of each text term is **binary**, only then is the below a probability measure.  \n",
        "If another feature method is used, such as BoW or TF/IDF, then the below is not the probability, but a proxy of it.  "
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "QwHZPJwuAssf"
      },
      "outputs": [],
      "source": [
        "P_class = sorted([[c, np.mean(dataset_feat_eng[\"_label_\"] == c)] for c in set(means_by_class.columns)])\n",
        "P_feature = sorted([[f, np.mean(dataset_feat_eng[f] > 0)] for f in dataset_feat_eng.columns if f != \"_label_\"])\n",
        "P_feature_inv = [[f, 1/p] for f, p in P_feature]\n",
        "\n",
        "P_class_arr = np.array(P_class)\n",
        "P_feature_arr = np.array(P_feature)\n",
        "P_feature_inv_arr = np.array(P_feature_inv)\n",
        "# Multiplying a \"column vector\" of feature probablities with a \"line vector\" of\n",
        "# class probilities to get a matrix where each element is a product of probabilities:\n",
        "P_class_prod_P_feature_inv_arr = np.outer(P_feature_inv_arr[:, 1].astype(float), P_class_arr[:, 1].astype(float))\n",
        "\n",
        "P_class_given_feature = means_by_class.copy()\n",
        "for feature_counter in range(len(P_class_given_feature)):\n",
        "  for c in P_class_given_feature.columns:\n",
        "    # Right hand side: P(feature | class) / P(feature)\n",
        "    P_class_given_feature[c][feature_counter] = means_by_class[c][feature_counter] / P_feature_arr[feature_counter, 1].astype(float)\n",
        "\n",
        "\n"
      ]
    },
    {
      "attachments": {},
      "cell_type": "markdown",
      "metadata": {
        "id": "ipfL9o6XX8pE"
      },
      "source": [
        "**The terms that are most indicative of class \"0\":**"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 363
        },
        "id": "bEl4iKwaX82G",
        "outputId": "2592018a-b404-469b-9d6c-feb27372506c"
      },
      "outputs": [
        {
          "data": {
            "application/vnd.google.colaboratory.intrinsic+json": {
              "summary": "{\n  \"name\": \"P_class_given_feature\",\n  \"rows\": 10,\n  \"fields\": [\n    {\n      \"column\": 0,\n      \"properties\": {\n        \"dtype\": \"number\",\n        \"std\": 0.06446778002122024,\n        \"min\": 1.4922600619195048,\n        \"max\": 1.685376069932617,\n        \"num_unique_values\": 8,\n        \"samples\": [\n          1.652964991664682,\n          1.5152179090259585,\n          1.685376069932617\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": 1,\n      \"properties\": {\n        \"dtype\": \"number\",\n        \"std\": 0.13600302988379048,\n        \"min\": 0.0,\n        \"max\": 0.4176173285198556,\n        \"num_unique_values\": 4,\n        \"samples\": [\n          0.09321815368746778,\n          0.1740072202166065,\n          0.0\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    }\n  ]\n}",
              "type": "dataframe"
            },
            "text/html": [
              "\n",
              "  <div id=\"df-f0024669-c92a-40dd-8712-14fdbc5d8347\" class=\"colab-df-container\">\n",
              "    <div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th>_label_</th>\n",
              "      <th>0</th>\n",
              "      <th>1</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>associated lower</th>\n",
              "      <td>1.685376</td>\n",
              "      <td>0.000000</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>metaanalysis</th>\n",
              "      <td>1.652965</td>\n",
              "      <td>0.000000</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>thirteen</th>\n",
              "      <td>1.566873</td>\n",
              "      <td>0.093218</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>twentythree</th>\n",
              "      <td>1.566873</td>\n",
              "      <td>0.000000</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>citation</th>\n",
              "      <td>1.566873</td>\n",
              "      <td>0.000000</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>median</th>\n",
              "      <td>1.525421</td>\n",
              "      <td>0.000000</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>verbal</th>\n",
              "      <td>1.515218</td>\n",
              "      <td>0.000000</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>blood pressure</th>\n",
              "      <td>1.515218</td>\n",
              "      <td>0.000000</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>hivone</th>\n",
              "      <td>1.504198</td>\n",
              "      <td>0.417617</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>abnormal</th>\n",
              "      <td>1.492260</td>\n",
              "      <td>0.174007</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>\n",
              "    <div class=\"colab-df-buttons\">\n",
              "\n",
              "  <div class=\"colab-df-container\">\n",
              "    <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-f0024669-c92a-40dd-8712-14fdbc5d8347')\"\n",
              "            title=\"Convert this dataframe to an interactive table.\"\n",
              "            style=\"display:none;\">\n",
              "\n",
              "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\" viewBox=\"0 -960 960 960\">\n",
              "    <path d=\"M120-120v-720h720v720H120Zm60-500h600v-160H180v160Zm220 220h160v-160H400v160Zm0 220h160v-160H400v160ZM180-400h160v-160H180v160Zm440 0h160v-160H620v160ZM180-180h160v-160H180v160Zm440 0h160v-160H620v160Z\"/>\n",
              "  </svg>\n",
              "    </button>\n",
              "\n",
              "  <style>\n",
              "    .colab-df-container {\n",
              "      display:flex;\n",
              "      gap: 12px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert {\n",
              "      background-color: #E8F0FE;\n",
              "      border: none;\n",
              "      border-radius: 50%;\n",
              "      cursor: pointer;\n",
              "      display: none;\n",
              "      fill: #1967D2;\n",
              "      height: 32px;\n",
              "      padding: 0 0 0 0;\n",
              "      width: 32px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert:hover {\n",
              "      background-color: #E2EBFA;\n",
              "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "      fill: #174EA6;\n",
              "    }\n",
              "\n",
              "    .colab-df-buttons div {\n",
              "      margin-bottom: 4px;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert {\n",
              "      background-color: #3B4455;\n",
              "      fill: #D2E3FC;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert:hover {\n",
              "      background-color: #434B5C;\n",
              "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
              "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
              "      fill: #FFFFFF;\n",
              "    }\n",
              "  </style>\n",
              "\n",
              "    <script>\n",
              "      const buttonEl =\n",
              "        document.querySelector('#df-f0024669-c92a-40dd-8712-14fdbc5d8347 button.colab-df-convert');\n",
              "      buttonEl.style.display =\n",
              "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "\n",
              "      async function convertToInteractive(key) {\n",
              "        const element = document.querySelector('#df-f0024669-c92a-40dd-8712-14fdbc5d8347');\n",
              "        const dataTable =\n",
              "          await google.colab.kernel.invokeFunction('convertToInteractive',\n",
              "                                                    [key], {});\n",
              "        if (!dataTable) return;\n",
              "\n",
              "        const docLinkHtml = 'Like what you see? Visit the ' +\n",
              "          '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
              "          + ' to learn more about interactive tables.';\n",
              "        element.innerHTML = '';\n",
              "        dataTable['output_type'] = 'display_data';\n",
              "        await google.colab.output.renderOutput(dataTable, element);\n",
              "        const docLink = document.createElement('div');\n",
              "        docLink.innerHTML = docLinkHtml;\n",
              "        element.appendChild(docLink);\n",
              "      }\n",
              "    </script>\n",
              "  </div>\n",
              "\n",
              "\n",
              "<div id=\"df-1dfdaf5c-5709-4f92-b8e3-1f6e22c3a1b1\">\n",
              "  <button class=\"colab-df-quickchart\" onclick=\"quickchart('df-1dfdaf5c-5709-4f92-b8e3-1f6e22c3a1b1')\"\n",
              "            title=\"Suggest charts\"\n",
              "            style=\"display:none;\">\n",
              "\n",
              "<svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
              "     width=\"24px\">\n",
              "    <g>\n",
              "        <path d=\"M19 3H5c-1.1 0-2 .9-2 2v14c0 1.1.9 2 2 2h14c1.1 0 2-.9 2-2V5c0-1.1-.9-2-2-2zM9 17H7v-7h2v7zm4 0h-2V7h2v10zm4 0h-2v-4h2v4z\"/>\n",
              "    </g>\n",
              "</svg>\n",
              "  </button>\n",
              "\n",
              "<style>\n",
              "  .colab-df-quickchart {\n",
              "      --bg-color: #E8F0FE;\n",
              "      --fill-color: #1967D2;\n",
              "      --hover-bg-color: #E2EBFA;\n",
              "      --hover-fill-color: #174EA6;\n",
              "      --disabled-fill-color: #AAA;\n",
              "      --disabled-bg-color: #DDD;\n",
              "  }\n",
              "\n",
              "  [theme=dark] .colab-df-quickchart {\n",
              "      --bg-color: #3B4455;\n",
              "      --fill-color: #D2E3FC;\n",
              "      --hover-bg-color: #434B5C;\n",
              "      --hover-fill-color: #FFFFFF;\n",
              "      --disabled-bg-color: #3B4455;\n",
              "      --disabled-fill-color: #666;\n",
              "  }\n",
              "\n",
              "  .colab-df-quickchart {\n",
              "    background-color: var(--bg-color);\n",
              "    border: none;\n",
              "    border-radius: 50%;\n",
              "    cursor: pointer;\n",
              "    display: none;\n",
              "    fill: var(--fill-color);\n",
              "    height: 32px;\n",
              "    padding: 0;\n",
              "    width: 32px;\n",
              "  }\n",
              "\n",
              "  .colab-df-quickchart:hover {\n",
              "    background-color: var(--hover-bg-color);\n",
              "    box-shadow: 0 1px 2px rgba(60, 64, 67, 0.3), 0 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "    fill: var(--button-hover-fill-color);\n",
              "  }\n",
              "\n",
              "  .colab-df-quickchart-complete:disabled,\n",
              "  .colab-df-quickchart-complete:disabled:hover {\n",
              "    background-color: var(--disabled-bg-color);\n",
              "    fill: var(--disabled-fill-color);\n",
              "    box-shadow: none;\n",
              "  }\n",
              "\n",
              "  .colab-df-spinner {\n",
              "    border: 2px solid var(--fill-color);\n",
              "    border-color: transparent;\n",
              "    border-bottom-color: var(--fill-color);\n",
              "    animation:\n",
              "      spin 1s steps(1) infinite;\n",
              "  }\n",
              "\n",
              "  @keyframes spin {\n",
              "    0% {\n",
              "      border-color: transparent;\n",
              "      border-bottom-color: var(--fill-color);\n",
              "      border-left-color: var(--fill-color);\n",
              "    }\n",
              "    20% {\n",
              "      border-color: transparent;\n",
              "      border-left-color: var(--fill-color);\n",
              "      border-top-color: var(--fill-color);\n",
              "    }\n",
              "    30% {\n",
              "      border-color: transparent;\n",
              "      border-left-color: var(--fill-color);\n",
              "      border-top-color: var(--fill-color);\n",
              "      border-right-color: var(--fill-color);\n",
              "    }\n",
              "    40% {\n",
              "      border-color: transparent;\n",
              "      border-right-color: var(--fill-color);\n",
              "      border-top-color: var(--fill-color);\n",
              "    }\n",
              "    60% {\n",
              "      border-color: transparent;\n",
              "      border-right-color: var(--fill-color);\n",
              "    }\n",
              "    80% {\n",
              "      border-color: transparent;\n",
              "      border-right-color: var(--fill-color);\n",
              "      border-bottom-color: var(--fill-color);\n",
              "    }\n",
              "    90% {\n",
              "      border-color: transparent;\n",
              "      border-bottom-color: var(--fill-color);\n",
              "    }\n",
              "  }\n",
              "</style>\n",
              "\n",
              "  <script>\n",
              "    async function quickchart(key) {\n",
              "      const quickchartButtonEl =\n",
              "        document.querySelector('#' + key + ' button');\n",
              "      quickchartButtonEl.disabled = true;  // To prevent multiple clicks.\n",
              "      quickchartButtonEl.classList.add('colab-df-spinner');\n",
              "      try {\n",
              "        const charts = await google.colab.kernel.invokeFunction(\n",
              "            'suggestCharts', [key], {});\n",
              "      } catch (error) {\n",
              "        console.error('Error during call to suggestCharts:', error);\n",
              "      }\n",
              "      quickchartButtonEl.classList.remove('colab-df-spinner');\n",
              "      quickchartButtonEl.classList.add('colab-df-quickchart-complete');\n",
              "    }\n",
              "    (() => {\n",
              "      let quickchartButtonEl =\n",
              "        document.querySelector('#df-1dfdaf5c-5709-4f92-b8e3-1f6e22c3a1b1 button');\n",
              "      quickchartButtonEl.style.display =\n",
              "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "    })();\n",
              "  </script>\n",
              "</div>\n",
              "\n",
              "    </div>\n",
              "  </div>\n"
            ],
            "text/plain": [
              "_label_                  0         1\n",
              "associated lower  1.685376  0.000000\n",
              "metaanalysis      1.652965  0.000000\n",
              "thirteen          1.566873  0.093218\n",
              "twentythree       1.566873  0.000000\n",
              "citation          1.566873  0.000000\n",
              "median            1.525421  0.000000\n",
              "verbal            1.515218  0.000000\n",
              "blood pressure    1.515218  0.000000\n",
              "hivone            1.504198  0.417617\n",
              "abnormal          1.492260  0.174007"
            ]
          },
          "execution_count": 21,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "P_class_given_feature.sort_values([0], ascending=False).head(10)"
      ]
    },
    {
      "attachments": {},
      "cell_type": "markdown",
      "metadata": {
        "id": "MaZFPdu0ZZgW"
      },
      "source": [
        "**The terms that are most indicative of class \"1\":**"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 363
        },
        "id": "Hqyvb9KlZZyW",
        "outputId": "f38626f8-b301-45b8-a731-c602c73cbb9e"
      },
      "outputs": [
        {
          "data": {
            "application/vnd.google.colaboratory.intrinsic+json": {
              "summary": "{\n  \"name\": \"P_class_given_feature\",\n  \"rows\": 10,\n  \"fields\": [\n    {\n      \"column\": 0,\n      \"properties\": {\n        \"dtype\": \"number\",\n        \"std\": 0.17405637458943246,\n        \"min\": 0.0,\n        \"max\": 0.5036377708978328,\n        \"num_unique_values\": 10,\n        \"samples\": [\n          0.5036377708978328,\n          0.06174869221735881,\n          0.0\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": 1,\n      \"properties\": {\n        \"dtype\": \"number\",\n        \"std\": 0.2818408072040074,\n        \"min\": 3.2626353790613716,\n        \"max\": 4.041458017934087,\n        \"num_unique_values\": 10,\n        \"samples\": [\n          3.262635379061372,\n          3.9601643221710443,\n          3.579577101598762\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    }\n  ]\n}",
              "type": "dataframe"
            },
            "text/html": [
              "\n",
              "  <div id=\"df-eb987d14-c456-4a2a-a1a3-9532b2330c9b\" class=\"colab-df-container\">\n",
              "    <div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th>_label_</th>\n",
              "      <th>0</th>\n",
              "      <th>1</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>promising</th>\n",
              "      <td>0.231060</td>\n",
              "      <td>4.041458</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>support use</th>\n",
              "      <td>0.061749</td>\n",
              "      <td>3.960164</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>option</th>\n",
              "      <td>0.030874</td>\n",
              "      <td>3.735155</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>alternative</th>\n",
              "      <td>0.096505</td>\n",
              "      <td>3.719795</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>tool</th>\n",
              "      <td>0.238762</td>\n",
              "      <td>3.712154</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>treatment option</th>\n",
              "      <td>0.000000</td>\n",
              "      <td>3.579577</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>treat</th>\n",
              "      <td>0.223839</td>\n",
              "      <td>3.480144</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>safe</th>\n",
              "      <td>0.344368</td>\n",
              "      <td>3.279367</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>findings support</th>\n",
              "      <td>0.503638</td>\n",
              "      <td>3.262635</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>suitable</th>\n",
              "      <td>0.447678</td>\n",
              "      <td>3.262635</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>\n",
              "    <div class=\"colab-df-buttons\">\n",
              "\n",
              "  <div class=\"colab-df-container\">\n",
              "    <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-eb987d14-c456-4a2a-a1a3-9532b2330c9b')\"\n",
              "            title=\"Convert this dataframe to an interactive table.\"\n",
              "            style=\"display:none;\">\n",
              "\n",
              "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\" viewBox=\"0 -960 960 960\">\n",
              "    <path d=\"M120-120v-720h720v720H120Zm60-500h600v-160H180v160Zm220 220h160v-160H400v160Zm0 220h160v-160H400v160ZM180-400h160v-160H180v160Zm440 0h160v-160H620v160ZM180-180h160v-160H180v160Zm440 0h160v-160H620v160Z\"/>\n",
              "  </svg>\n",
              "    </button>\n",
              "\n",
              "  <style>\n",
              "    .colab-df-container {\n",
              "      display:flex;\n",
              "      gap: 12px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert {\n",
              "      background-color: #E8F0FE;\n",
              "      border: none;\n",
              "      border-radius: 50%;\n",
              "      cursor: pointer;\n",
              "      display: none;\n",
              "      fill: #1967D2;\n",
              "      height: 32px;\n",
              "      padding: 0 0 0 0;\n",
              "      width: 32px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert:hover {\n",
              "      background-color: #E2EBFA;\n",
              "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "      fill: #174EA6;\n",
              "    }\n",
              "\n",
              "    .colab-df-buttons div {\n",
              "      margin-bottom: 4px;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert {\n",
              "      background-color: #3B4455;\n",
              "      fill: #D2E3FC;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert:hover {\n",
              "      background-color: #434B5C;\n",
              "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
              "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
              "      fill: #FFFFFF;\n",
              "    }\n",
              "  </style>\n",
              "\n",
              "    <script>\n",
              "      const buttonEl =\n",
              "        document.querySelector('#df-eb987d14-c456-4a2a-a1a3-9532b2330c9b button.colab-df-convert');\n",
              "      buttonEl.style.display =\n",
              "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "\n",
              "      async function convertToInteractive(key) {\n",
              "        const element = document.querySelector('#df-eb987d14-c456-4a2a-a1a3-9532b2330c9b');\n",
              "        const dataTable =\n",
              "          await google.colab.kernel.invokeFunction('convertToInteractive',\n",
              "                                                    [key], {});\n",
              "        if (!dataTable) return;\n",
              "\n",
              "        const docLinkHtml = 'Like what you see? Visit the ' +\n",
              "          '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
              "          + ' to learn more about interactive tables.';\n",
              "        element.innerHTML = '';\n",
              "        dataTable['output_type'] = 'display_data';\n",
              "        await google.colab.output.renderOutput(dataTable, element);\n",
              "        const docLink = document.createElement('div');\n",
              "        docLink.innerHTML = docLinkHtml;\n",
              "        element.appendChild(docLink);\n",
              "      }\n",
              "    </script>\n",
              "  </div>\n",
              "\n",
              "\n",
              "<div id=\"df-d4e0cb72-f0ee-4013-965f-5fa38f21127a\">\n",
              "  <button class=\"colab-df-quickchart\" onclick=\"quickchart('df-d4e0cb72-f0ee-4013-965f-5fa38f21127a')\"\n",
              "            title=\"Suggest charts\"\n",
              "            style=\"display:none;\">\n",
              "\n",
              "<svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
              "     width=\"24px\">\n",
              "    <g>\n",
              "        <path d=\"M19 3H5c-1.1 0-2 .9-2 2v14c0 1.1.9 2 2 2h14c1.1 0 2-.9 2-2V5c0-1.1-.9-2-2-2zM9 17H7v-7h2v7zm4 0h-2V7h2v10zm4 0h-2v-4h2v4z\"/>\n",
              "    </g>\n",
              "</svg>\n",
              "  </button>\n",
              "\n",
              "<style>\n",
              "  .colab-df-quickchart {\n",
              "      --bg-color: #E8F0FE;\n",
              "      --fill-color: #1967D2;\n",
              "      --hover-bg-color: #E2EBFA;\n",
              "      --hover-fill-color: #174EA6;\n",
              "      --disabled-fill-color: #AAA;\n",
              "      --disabled-bg-color: #DDD;\n",
              "  }\n",
              "\n",
              "  [theme=dark] .colab-df-quickchart {\n",
              "      --bg-color: #3B4455;\n",
              "      --fill-color: #D2E3FC;\n",
              "      --hover-bg-color: #434B5C;\n",
              "      --hover-fill-color: #FFFFFF;\n",
              "      --disabled-bg-color: #3B4455;\n",
              "      --disabled-fill-color: #666;\n",
              "  }\n",
              "\n",
              "  .colab-df-quickchart {\n",
              "    background-color: var(--bg-color);\n",
              "    border: none;\n",
              "    border-radius: 50%;\n",
              "    cursor: pointer;\n",
              "    display: none;\n",
              "    fill: var(--fill-color);\n",
              "    height: 32px;\n",
              "    padding: 0;\n",
              "    width: 32px;\n",
              "  }\n",
              "\n",
              "  .colab-df-quickchart:hover {\n",
              "    background-color: var(--hover-bg-color);\n",
              "    box-shadow: 0 1px 2px rgba(60, 64, 67, 0.3), 0 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "    fill: var(--button-hover-fill-color);\n",
              "  }\n",
              "\n",
              "  .colab-df-quickchart-complete:disabled,\n",
              "  .colab-df-quickchart-complete:disabled:hover {\n",
              "    background-color: var(--disabled-bg-color);\n",
              "    fill: var(--disabled-fill-color);\n",
              "    box-shadow: none;\n",
              "  }\n",
              "\n",
              "  .colab-df-spinner {\n",
              "    border: 2px solid var(--fill-color);\n",
              "    border-color: transparent;\n",
              "    border-bottom-color: var(--fill-color);\n",
              "    animation:\n",
              "      spin 1s steps(1) infinite;\n",
              "  }\n",
              "\n",
              "  @keyframes spin {\n",
              "    0% {\n",
              "      border-color: transparent;\n",
              "      border-bottom-color: var(--fill-color);\n",
              "      border-left-color: var(--fill-color);\n",
              "    }\n",
              "    20% {\n",
              "      border-color: transparent;\n",
              "      border-left-color: var(--fill-color);\n",
              "      border-top-color: var(--fill-color);\n",
              "    }\n",
              "    30% {\n",
              "      border-color: transparent;\n",
              "      border-left-color: var(--fill-color);\n",
              "      border-top-color: var(--fill-color);\n",
              "      border-right-color: var(--fill-color);\n",
              "    }\n",
              "    40% {\n",
              "      border-color: transparent;\n",
              "      border-right-color: var(--fill-color);\n",
              "      border-top-color: var(--fill-color);\n",
              "    }\n",
              "    60% {\n",
              "      border-color: transparent;\n",
              "      border-right-color: var(--fill-color);\n",
              "    }\n",
              "    80% {\n",
              "      border-color: transparent;\n",
              "      border-right-color: var(--fill-color);\n",
              "      border-bottom-color: var(--fill-color);\n",
              "    }\n",
              "    90% {\n",
              "      border-color: transparent;\n",
              "      border-bottom-color: var(--fill-color);\n",
              "    }\n",
              "  }\n",
              "</style>\n",
              "\n",
              "  <script>\n",
              "    async function quickchart(key) {\n",
              "      const quickchartButtonEl =\n",
              "        document.querySelector('#' + key + ' button');\n",
              "      quickchartButtonEl.disabled = true;  // To prevent multiple clicks.\n",
              "      quickchartButtonEl.classList.add('colab-df-spinner');\n",
              "      try {\n",
              "        const charts = await google.colab.kernel.invokeFunction(\n",
              "            'suggestCharts', [key], {});\n",
              "      } catch (error) {\n",
              "        console.error('Error during call to suggestCharts:', error);\n",
              "      }\n",
              "      quickchartButtonEl.classList.remove('colab-df-spinner');\n",
              "      quickchartButtonEl.classList.add('colab-df-quickchart-complete');\n",
              "    }\n",
              "    (() => {\n",
              "      let quickchartButtonEl =\n",
              "        document.querySelector('#df-d4e0cb72-f0ee-4013-965f-5fa38f21127a button');\n",
              "      quickchartButtonEl.style.display =\n",
              "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "    })();\n",
              "  </script>\n",
              "</div>\n",
              "\n",
              "    </div>\n",
              "  </div>\n"
            ],
            "text/plain": [
              "_label_                  0         1\n",
              "promising         0.231060  4.041458\n",
              "support use       0.061749  3.960164\n",
              "option            0.030874  3.735155\n",
              "alternative       0.096505  3.719795\n",
              "tool              0.238762  3.712154\n",
              "treatment option  0.000000  3.579577\n",
              "treat             0.223839  3.480144\n",
              "safe              0.344368  3.279367\n",
              "findings support  0.503638  3.262635\n",
              "suitable          0.447678  3.262635"
            ]
          },
          "execution_count": 22,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "P_class_given_feature.sort_values([1], ascending=False).head(10)"
      ]
    },
    {
      "attachments": {},
      "cell_type": "markdown",
      "metadata": {
        "id": "YyHadLWYchW7"
      },
      "source": [
        "## Feature selection\n",
        "This is a univariate feature selection process.  \n",
        "It is based on conditional dependency between a feature being 0/1 and a class being 0/1, thus the mean value of the feature is its probability.  \n",
        "Note that the process of feature selection is done **on the training set**.   \n"
      ]
    },
    {
      "attachments": {},
      "cell_type": "markdown",
      "metadata": {
        "id": "glryYgkh9uGr"
      },
      "source": [
        "For each class, choose the most indicative features.  \n",
        "Either maximize the:   \n",
        "* a-priori distribution P(feature | class), Max Liklihood  \n",
        "or  \n",
        "* a posteriori P(class | feature), MAP"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "bTnI3Lpk95BU"
      },
      "outputs": [],
      "source": [
        "chosen_features = []\n",
        "if config_dict[\"maximize_a_priori\"] == True:\n",
        "  classes = means_by_class.columns\n",
        "  for c in classes:\n",
        "    chosen_features += list(means_by_class[c].sort_values(ascending=False).index[:config_dict[\"num_chosen_features_per_class\"] + 1])\n",
        "else:\n",
        "  classes = P_class_given_feature.columns\n",
        "  for c in classes:\n",
        "    chosen_features += list(P_class_given_feature[c].sort_values(ascending=False).index[:config_dict[\"num_chosen_features_per_class\"] + 1])\n",
        "\n",
        "\n",
        "chosen_features = list(set(chosen_features))"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "QsRkvKXuAApE",
        "outputId": "73d28095-c733-423c-a200-6eab10416429"
      },
      "outputs": [
        {
          "data": {
            "text/plain": [
              "['mice',\n",
              " 'efficacy',\n",
              " 'et',\n",
              " 'positively',\n",
              " 'appeared',\n",
              " 'investigations',\n",
              " 'preliminary',\n",
              " 'pediatric',\n",
              " 'burden',\n",
              " 'reducing',\n",
              " 'strategy',\n",
              " 'family',\n",
              " 'educational',\n",
              " 'retrospective',\n",
              " 'treatments',\n",
              " 'provided',\n",
              " 'education',\n",
              " 'zeroone',\n",
              " 'ninetyseven',\n",
              " 'additional',\n",
              " 'target',\n",
              " 'standard',\n",
              " 'crs type',\n",
              " 'memory',\n",
              " 'therapeutic',\n",
              " 'sample size',\n",
              " 'cancer risk',\n",
              " 'prevent',\n",
              " 'settings',\n",
              " 'ratio',\n",
              " 'associated increased',\n",
              " 'optimal',\n",
              " 'bone',\n",
              " 'treatment option',\n",
              " 'taken',\n",
              " 'needs',\n",
              " 'used',\n",
              " 'effectiveness',\n",
              " 'recommendations',\n",
              " 'barriers',\n",
              " 'death',\n",
              " 'parents',\n",
              " 'twentyone',\n",
              " 'comprehensive',\n",
              " 'adiposity',\n",
              " 'younger',\n",
              " 'treat',\n",
              " 'distribution',\n",
              " 'provide',\n",
              " 'crc',\n",
              " 'metaanalysis',\n",
              " 'appears',\n",
              " 'therapy',\n",
              " 'procedures',\n",
              " 'underwent',\n",
              " 'safe',\n",
              " 'choice',\n",
              " 'tool',\n",
              " 'improve',\n",
              " 'degree',\n",
              " 'suitable',\n",
              " 'statistically significant',\n",
              " 'absence',\n",
              " 'key',\n",
              " 'malnutrition',\n",
              " 'especially',\n",
              " 'days',\n",
              " 'consistent',\n",
              " 'cell',\n",
              " 'correlated',\n",
              " 'twentythree',\n",
              " 'direct',\n",
              " 'genotype',\n",
              " 'nctone million',\n",
              " 'supports',\n",
              " 'professionals',\n",
              " 'performing',\n",
              " 'valuable',\n",
              " 'received',\n",
              " 'improving',\n",
              " 'findings',\n",
              " 'expected',\n",
              " 'density',\n",
              " 'cervical',\n",
              " 'psychological',\n",
              " 'feasible',\n",
              " 'evidencebased',\n",
              " 'values',\n",
              " 'develop',\n",
              " 'results provide',\n",
              " 'clinical practice',\n",
              " 'high risk',\n",
              " 'findings suggest',\n",
              " 'help',\n",
              " 'thirty',\n",
              " 'policy',\n",
              " 'conization',\n",
              " 'simple',\n",
              " 'examined',\n",
              " 'major',\n",
              " 'showed',\n",
              " 'hiv',\n",
              " 'training',\n",
              " 'nineteen',\n",
              " 'new',\n",
              " 'variation',\n",
              " 'able',\n",
              " 'respectively',\n",
              " 'cohort',\n",
              " 'original',\n",
              " 'intraoperative',\n",
              " 'intensive',\n",
              " 'visual',\n",
              " 'al',\n",
              " 'pathways',\n",
              " 'literature',\n",
              " 'providing',\n",
              " 'sixteen',\n",
              " 'decreased',\n",
              " 'benefit',\n",
              " 'seventy',\n",
              " 'provides',\n",
              " 'mean',\n",
              " 'compared',\n",
              " 'suggests',\n",
              " 'cytokines',\n",
              " 'example',\n",
              " 'higher',\n",
              " 'efforts',\n",
              " 'supported',\n",
              " 'cohorts',\n",
              " 'evaluation',\n",
              " 'series',\n",
              " 'occur',\n",
              " 'prophylaxis',\n",
              " 'longitudinal',\n",
              " 'proportion',\n",
              " 'counseling',\n",
              " 'individual',\n",
              " 'et al',\n",
              " 'cells',\n",
              " 'procedure',\n",
              " 'adult',\n",
              " 'important',\n",
              " 'students',\n",
              " 'identification',\n",
              " 'useful',\n",
              " 'million',\n",
              " 'short',\n",
              " 'crs',\n",
              " 'adolescents',\n",
              " 'analgesia',\n",
              " 'technique',\n",
              " 'expression',\n",
              " 'recommend',\n",
              " 'clinicians',\n",
              " 'appropriate',\n",
              " 'findings provide',\n",
              " 'developing',\n",
              " 'incidence',\n",
              " 'rehabilitation',\n",
              " 'assessment',\n",
              " 'citation',\n",
              " 'management',\n",
              " 'detected',\n",
              " 'adherence',\n",
              " 'associated lower',\n",
              " 'subjects',\n",
              " 'measure',\n",
              " 'make',\n",
              " 'angiogenin',\n",
              " 'learning',\n",
              " 'seven',\n",
              " 'frequently',\n",
              " 'peripheral',\n",
              " 'relevant',\n",
              " 'sample',\n",
              " 'techniques',\n",
              " 'types',\n",
              " 'program',\n",
              " 'treating',\n",
              " 'million thousand',\n",
              " 'nctone',\n",
              " 'twentyfive',\n",
              " 'order',\n",
              " 'measurement',\n",
              " 'pathway',\n",
              " 'association',\n",
              " 'lifestyle',\n",
              " 'tnf',\n",
              " 'effectively',\n",
              " 'grssixtythree',\n",
              " 'snps',\n",
              " 'score',\n",
              " 'mechanism',\n",
              " 'facilitate',\n",
              " 'interval',\n",
              " 'fourteen',\n",
              " 'invasive',\n",
              " 'approach',\n",
              " 'bmd',\n",
              " 'opioid',\n",
              " 'identified',\n",
              " 'attention',\n",
              " 'hivone',\n",
              " 'day',\n",
              " 'approaches',\n",
              " 'cost',\n",
              " 'highly',\n",
              " 'tested',\n",
              " 'considering',\n",
              " 'genes',\n",
              " 'significantly associated',\n",
              " 'cardiovascular risk',\n",
              " 'active commuting',\n",
              " 'registered',\n",
              " 'maintain',\n",
              " 'represents',\n",
              " 'caused',\n",
              " 'combination',\n",
              " 'account',\n",
              " 'avoid',\n",
              " 'controls',\n",
              " 'eighteen',\n",
              " 'considered',\n",
              " 'surgical',\n",
              " 'area',\n",
              " 'achieve',\n",
              " 'susceptibility',\n",
              " 'models',\n",
              " 'support',\n",
              " 'larger',\n",
              " 'importance',\n",
              " 'effective',\n",
              " 'report',\n",
              " 'application',\n",
              " 'rats',\n",
              " 'replication',\n",
              " 'medication',\n",
              " 'availability',\n",
              " 'require',\n",
              " 'future studies',\n",
              " 'years',\n",
              " 'relation',\n",
              " 'included',\n",
              " 'caution',\n",
              " 'prevalence',\n",
              " 'significance',\n",
              " 'according',\n",
              " 'tumor',\n",
              " 'symptoms',\n",
              " 'verbal',\n",
              " 'ldl',\n",
              " 'methylation',\n",
              " 'medical',\n",
              " 'recommended',\n",
              " 'highrisk',\n",
              " 'furthermore',\n",
              " 'consideration',\n",
              " 'acceptable',\n",
              " 'services',\n",
              " 'intervention',\n",
              " 'age',\n",
              " 'initial',\n",
              " 'thousand',\n",
              " 'blood pressure',\n",
              " 'conducted',\n",
              " 'prevention',\n",
              " 'targeting',\n",
              " 'advantages',\n",
              " 'health',\n",
              " 'birth',\n",
              " 'physicians',\n",
              " 'ninetyfive',\n",
              " 'associated',\n",
              " 'cannabis',\n",
              " 'defined',\n",
              " 'reduces',\n",
              " 'promote',\n",
              " 'human',\n",
              " 'promising',\n",
              " 'resistance',\n",
              " 'particularly',\n",
              " 'interventions',\n",
              " 'routine',\n",
              " 'reduce',\n",
              " 'thirteen',\n",
              " 'exercise',\n",
              " 'chemotherapy',\n",
              " 'hypothesis',\n",
              " 'abnormal',\n",
              " 'significant difference',\n",
              " 'home',\n",
              " 'public health',\n",
              " 'targeted',\n",
              " 'given',\n",
              " 'nutritional',\n",
              " 'median',\n",
              " 'dicerone',\n",
              " 'condition',\n",
              " 'involving',\n",
              " 'socioeconomic',\n",
              " 'public',\n",
              " 'schizophrenia',\n",
              " 'therapies',\n",
              " 'finally',\n",
              " 'implementation',\n",
              " 'thickness',\n",
              " 'later',\n",
              " 'identify',\n",
              " 'option',\n",
              " 'method',\n",
              " 'metastatic',\n",
              " 'commuting',\n",
              " 'anesthesia',\n",
              " 'treatment',\n",
              " 'associated higher',\n",
              " 'good',\n",
              " 'beneficial',\n",
              " 'difference',\n",
              " 'perform',\n",
              " 'polymorphism',\n",
              " 'skills',\n",
              " 'involved',\n",
              " 'snp',\n",
              " 'wide',\n",
              " 'led',\n",
              " 'focus',\n",
              " 'figure',\n",
              " 'selection',\n",
              " 'use',\n",
              " 'alternative',\n",
              " 'highlights',\n",
              " 'methods',\n",
              " 'providers',\n",
              " 'scale',\n",
              " 'polymorphisms',\n",
              " 'regular',\n",
              " 'nutrition',\n",
              " 'address',\n",
              " 'strategies',\n",
              " 'reduced',\n",
              " 'bp',\n",
              " 'drug',\n",
              " 'metastases',\n",
              " 'finding',\n",
              " 'clinicaltrialsgov',\n",
              " 'practice',\n",
              " 'findings support',\n",
              " 'statistical',\n",
              " 'indicate',\n",
              " 'statistically',\n",
              " 'followed',\n",
              " 'recent',\n",
              " 'screening',\n",
              " 'resulted',\n",
              " 'preventive',\n",
              " 'metastasis',\n",
              " 'approximately',\n",
              " 'offer',\n",
              " 'cortical',\n",
              " 'context',\n",
              " 'occurred',\n",
              " 'cycles',\n",
              " 'explained',\n",
              " 'patients treated',\n",
              " 'thousand seven',\n",
              " 'twentyseven',\n",
              " 'doses',\n",
              " 'plasma',\n",
              " 'thirtyone',\n",
              " 'thirtythree',\n",
              " 'beliefs',\n",
              " 'samples',\n",
              " 'mass',\n",
              " 'twentysix',\n",
              " 'participants',\n",
              " 'oxygen',\n",
              " 'results support',\n",
              " 'identifying',\n",
              " 'mental health',\n",
              " 'fortytwo',\n",
              " 'support use',\n",
              " 'circulating',\n",
              " 'enhance',\n",
              " 'need',\n",
              " 'programs',\n",
              " 'studies needed',\n",
              " 'suggest',\n",
              " 'critical',\n",
              " 'previously',\n",
              " 'developed',\n",
              " 'maintenance',\n",
              " 'helpful',\n",
              " 'planning',\n",
              " 'valid',\n",
              " 'necessary',\n",
              " 'ninetyeight',\n",
              " 'independent',\n",
              " 'pressure',\n",
              " 'consider',\n",
              " 'monitoring']"
            ]
          },
          "execution_count": 24,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "chosen_features"
      ]
    },
    {
      "attachments": {},
      "cell_type": "markdown",
      "metadata": {
        "id": "vhgOQmjkBq78"
      },
      "source": [
        "### Leave only chosen features:\n",
        "Now that we deduced which features are \"important\" based on the train set, we select them for both the train set and the test set.  "
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 255
        },
        "id": "w4bY4gmWBrGe",
        "outputId": "7eb8a290-d5b3-45d0-8b58-6821b5654c8f"
      },
      "outputs": [
        {
          "data": {
            "application/vnd.google.colaboratory.intrinsic+json": {
              "type": "dataframe",
              "variable_name": "dataset_feat_eng_train_selected"
            },
            "text/html": [
              "\n",
              "  <div id=\"df-8e4900bd-ab9f-46cb-b8b0-25e80d9b2135\" class=\"colab-df-container\">\n",
              "    <div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>mice</th>\n",
              "      <th>efficacy</th>\n",
              "      <th>et</th>\n",
              "      <th>positively</th>\n",
              "      <th>appeared</th>\n",
              "      <th>investigations</th>\n",
              "      <th>preliminary</th>\n",
              "      <th>pediatric</th>\n",
              "      <th>burden</th>\n",
              "      <th>reducing</th>\n",
              "      <th>...</th>\n",
              "      <th>helpful</th>\n",
              "      <th>planning</th>\n",
              "      <th>valid</th>\n",
              "      <th>necessary</th>\n",
              "      <th>ninetyeight</th>\n",
              "      <th>independent</th>\n",
              "      <th>pressure</th>\n",
              "      <th>consider</th>\n",
              "      <th>monitoring</th>\n",
              "      <th>_label_</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>...</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>...</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>...</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3</th>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>...</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>4</th>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>...</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "<p>5 rows × 403 columns</p>\n",
              "</div>\n",
              "    <div class=\"colab-df-buttons\">\n",
              "\n",
              "  <div class=\"colab-df-container\">\n",
              "    <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-8e4900bd-ab9f-46cb-b8b0-25e80d9b2135')\"\n",
              "            title=\"Convert this dataframe to an interactive table.\"\n",
              "            style=\"display:none;\">\n",
              "\n",
              "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\" viewBox=\"0 -960 960 960\">\n",
              "    <path d=\"M120-120v-720h720v720H120Zm60-500h600v-160H180v160Zm220 220h160v-160H400v160Zm0 220h160v-160H400v160ZM180-400h160v-160H180v160Zm440 0h160v-160H620v160ZM180-180h160v-160H180v160Zm440 0h160v-160H620v160Z\"/>\n",
              "  </svg>\n",
              "    </button>\n",
              "\n",
              "  <style>\n",
              "    .colab-df-container {\n",
              "      display:flex;\n",
              "      gap: 12px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert {\n",
              "      background-color: #E8F0FE;\n",
              "      border: none;\n",
              "      border-radius: 50%;\n",
              "      cursor: pointer;\n",
              "      display: none;\n",
              "      fill: #1967D2;\n",
              "      height: 32px;\n",
              "      padding: 0 0 0 0;\n",
              "      width: 32px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert:hover {\n",
              "      background-color: #E2EBFA;\n",
              "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "      fill: #174EA6;\n",
              "    }\n",
              "\n",
              "    .colab-df-buttons div {\n",
              "      margin-bottom: 4px;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert {\n",
              "      background-color: #3B4455;\n",
              "      fill: #D2E3FC;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert:hover {\n",
              "      background-color: #434B5C;\n",
              "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
              "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
              "      fill: #FFFFFF;\n",
              "    }\n",
              "  </style>\n",
              "\n",
              "    <script>\n",
              "      const buttonEl =\n",
              "        document.querySelector('#df-8e4900bd-ab9f-46cb-b8b0-25e80d9b2135 button.colab-df-convert');\n",
              "      buttonEl.style.display =\n",
              "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "\n",
              "      async function convertToInteractive(key) {\n",
              "        const element = document.querySelector('#df-8e4900bd-ab9f-46cb-b8b0-25e80d9b2135');\n",
              "        const dataTable =\n",
              "          await google.colab.kernel.invokeFunction('convertToInteractive',\n",
              "                                                    [key], {});\n",
              "        if (!dataTable) return;\n",
              "\n",
              "        const docLinkHtml = 'Like what you see? Visit the ' +\n",
              "          '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
              "          + ' to learn more about interactive tables.';\n",
              "        element.innerHTML = '';\n",
              "        dataTable['output_type'] = 'display_data';\n",
              "        await google.colab.output.renderOutput(dataTable, element);\n",
              "        const docLink = document.createElement('div');\n",
              "        docLink.innerHTML = docLinkHtml;\n",
              "        element.appendChild(docLink);\n",
              "      }\n",
              "    </script>\n",
              "  </div>\n",
              "\n",
              "\n",
              "<div id=\"df-30150150-0e59-4224-a640-c5c766261c11\">\n",
              "  <button class=\"colab-df-quickchart\" onclick=\"quickchart('df-30150150-0e59-4224-a640-c5c766261c11')\"\n",
              "            title=\"Suggest charts\"\n",
              "            style=\"display:none;\">\n",
              "\n",
              "<svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
              "     width=\"24px\">\n",
              "    <g>\n",
              "        <path d=\"M19 3H5c-1.1 0-2 .9-2 2v14c0 1.1.9 2 2 2h14c1.1 0 2-.9 2-2V5c0-1.1-.9-2-2-2zM9 17H7v-7h2v7zm4 0h-2V7h2v10zm4 0h-2v-4h2v4z\"/>\n",
              "    </g>\n",
              "</svg>\n",
              "  </button>\n",
              "\n",
              "<style>\n",
              "  .colab-df-quickchart {\n",
              "      --bg-color: #E8F0FE;\n",
              "      --fill-color: #1967D2;\n",
              "      --hover-bg-color: #E2EBFA;\n",
              "      --hover-fill-color: #174EA6;\n",
              "      --disabled-fill-color: #AAA;\n",
              "      --disabled-bg-color: #DDD;\n",
              "  }\n",
              "\n",
              "  [theme=dark] .colab-df-quickchart {\n",
              "      --bg-color: #3B4455;\n",
              "      --fill-color: #D2E3FC;\n",
              "      --hover-bg-color: #434B5C;\n",
              "      --hover-fill-color: #FFFFFF;\n",
              "      --disabled-bg-color: #3B4455;\n",
              "      --disabled-fill-color: #666;\n",
              "  }\n",
              "\n",
              "  .colab-df-quickchart {\n",
              "    background-color: var(--bg-color);\n",
              "    border: none;\n",
              "    border-radius: 50%;\n",
              "    cursor: pointer;\n",
              "    display: none;\n",
              "    fill: var(--fill-color);\n",
              "    height: 32px;\n",
              "    padding: 0;\n",
              "    width: 32px;\n",
              "  }\n",
              "\n",
              "  .colab-df-quickchart:hover {\n",
              "    background-color: var(--hover-bg-color);\n",
              "    box-shadow: 0 1px 2px rgba(60, 64, 67, 0.3), 0 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "    fill: var(--button-hover-fill-color);\n",
              "  }\n",
              "\n",
              "  .colab-df-quickchart-complete:disabled,\n",
              "  .colab-df-quickchart-complete:disabled:hover {\n",
              "    background-color: var(--disabled-bg-color);\n",
              "    fill: var(--disabled-fill-color);\n",
              "    box-shadow: none;\n",
              "  }\n",
              "\n",
              "  .colab-df-spinner {\n",
              "    border: 2px solid var(--fill-color);\n",
              "    border-color: transparent;\n",
              "    border-bottom-color: var(--fill-color);\n",
              "    animation:\n",
              "      spin 1s steps(1) infinite;\n",
              "  }\n",
              "\n",
              "  @keyframes spin {\n",
              "    0% {\n",
              "      border-color: transparent;\n",
              "      border-bottom-color: var(--fill-color);\n",
              "      border-left-color: var(--fill-color);\n",
              "    }\n",
              "    20% {\n",
              "      border-color: transparent;\n",
              "      border-left-color: var(--fill-color);\n",
              "      border-top-color: var(--fill-color);\n",
              "    }\n",
              "    30% {\n",
              "      border-color: transparent;\n",
              "      border-left-color: var(--fill-color);\n",
              "      border-top-color: var(--fill-color);\n",
              "      border-right-color: var(--fill-color);\n",
              "    }\n",
              "    40% {\n",
              "      border-color: transparent;\n",
              "      border-right-color: var(--fill-color);\n",
              "      border-top-color: var(--fill-color);\n",
              "    }\n",
              "    60% {\n",
              "      border-color: transparent;\n",
              "      border-right-color: var(--fill-color);\n",
              "    }\n",
              "    80% {\n",
              "      border-color: transparent;\n",
              "      border-right-color: var(--fill-color);\n",
              "      border-bottom-color: var(--fill-color);\n",
              "    }\n",
              "    90% {\n",
              "      border-color: transparent;\n",
              "      border-bottom-color: var(--fill-color);\n",
              "    }\n",
              "  }\n",
              "</style>\n",
              "\n",
              "  <script>\n",
              "    async function quickchart(key) {\n",
              "      const quickchartButtonEl =\n",
              "        document.querySelector('#' + key + ' button');\n",
              "      quickchartButtonEl.disabled = true;  // To prevent multiple clicks.\n",
              "      quickchartButtonEl.classList.add('colab-df-spinner');\n",
              "      try {\n",
              "        const charts = await google.colab.kernel.invokeFunction(\n",
              "            'suggestCharts', [key], {});\n",
              "      } catch (error) {\n",
              "        console.error('Error during call to suggestCharts:', error);\n",
              "      }\n",
              "      quickchartButtonEl.classList.remove('colab-df-spinner');\n",
              "      quickchartButtonEl.classList.add('colab-df-quickchart-complete');\n",
              "    }\n",
              "    (() => {\n",
              "      let quickchartButtonEl =\n",
              "        document.querySelector('#df-30150150-0e59-4224-a640-c5c766261c11 button');\n",
              "      quickchartButtonEl.style.display =\n",
              "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "    })();\n",
              "  </script>\n",
              "</div>\n",
              "\n",
              "    </div>\n",
              "  </div>\n"
            ],
            "text/plain": [
              "   mice  efficacy  et  positively  appeared  investigations  preliminary  \\\n",
              "0     0         0   0           0         0               0            0   \n",
              "1     1         0   0           0         0               0            0   \n",
              "2     0         0   0           0         0               0            0   \n",
              "3     0         0   0           0         0               0            0   \n",
              "4     0         0   0           0         0               0            0   \n",
              "\n",
              "   pediatric  burden  reducing  ...  helpful  planning  valid  necessary  \\\n",
              "0          0       0         0  ...        0         0      0          0   \n",
              "1          0       0         0  ...        0         0      0          0   \n",
              "2          0       0         0  ...        1         0      0          0   \n",
              "3          0       0         0  ...        0         0      0          1   \n",
              "4          0       0         0  ...        0         0      0          0   \n",
              "\n",
              "   ninetyeight  independent  pressure  consider  monitoring  _label_  \n",
              "0            0            0         0         0           0        0  \n",
              "1            0            0         0         0           0        0  \n",
              "2            0            0         0         0           0        0  \n",
              "3            0            0         0         0           0        0  \n",
              "4            0            0         0         0           0        0  \n",
              "\n",
              "[5 rows x 403 columns]"
            ]
          },
          "execution_count": 25,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "dataset_feat_eng_train_selected = dataset_feat_eng_train.filter(chosen_features + [\"_label_\"])\n",
        "dataset_feat_eng_test_selected = dataset_feat_eng_test.filter(chosen_features + [\"_label_\"])\n",
        "\n",
        "dataset_feat_eng_train_selected.head()"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "5jiW-llRT4VK",
        "outputId": "b33b59e0-acc5-41d6-ec41-e06dfe67dc06"
      },
      "outputs": [
        {
          "data": {
            "text/plain": [
              "0    4845\n",
              "1    1662\n",
              "Name: _label_, dtype: int64"
            ]
          },
          "execution_count": 26,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "dataset_feat_eng_train_selected[\"_label_\"].value_counts()"
      ]
    },
    {
      "attachments": {},
      "cell_type": "markdown",
      "metadata": {
        "id": "4JAQspmJCwfT"
      },
      "source": [
        "# Machine Learning   \n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 255
        },
        "id": "QIVm-heY6AdA",
        "outputId": "83476c52-2560-4433-ba4f-3fdeaf7c9c9c"
      },
      "outputs": [
        {
          "data": {
            "application/vnd.google.colaboratory.intrinsic+json": {
              "type": "dataframe",
              "variable_name": "dataset_feat_eng_train_selected"
            },
            "text/html": [
              "\n",
              "  <div id=\"df-317a4622-8f22-4ce0-a364-a0ff2ec822ca\" class=\"colab-df-container\">\n",
              "    <div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>mice</th>\n",
              "      <th>efficacy</th>\n",
              "      <th>et</th>\n",
              "      <th>positively</th>\n",
              "      <th>appeared</th>\n",
              "      <th>investigations</th>\n",
              "      <th>preliminary</th>\n",
              "      <th>pediatric</th>\n",
              "      <th>burden</th>\n",
              "      <th>reducing</th>\n",
              "      <th>...</th>\n",
              "      <th>helpful</th>\n",
              "      <th>planning</th>\n",
              "      <th>valid</th>\n",
              "      <th>necessary</th>\n",
              "      <th>ninetyeight</th>\n",
              "      <th>independent</th>\n",
              "      <th>pressure</th>\n",
              "      <th>consider</th>\n",
              "      <th>monitoring</th>\n",
              "      <th>_label_</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>...</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>...</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>...</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3</th>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>...</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>4</th>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>...</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "<p>5 rows × 403 columns</p>\n",
              "</div>\n",
              "    <div class=\"colab-df-buttons\">\n",
              "\n",
              "  <div class=\"colab-df-container\">\n",
              "    <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-317a4622-8f22-4ce0-a364-a0ff2ec822ca')\"\n",
              "            title=\"Convert this dataframe to an interactive table.\"\n",
              "            style=\"display:none;\">\n",
              "\n",
              "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\" viewBox=\"0 -960 960 960\">\n",
              "    <path d=\"M120-120v-720h720v720H120Zm60-500h600v-160H180v160Zm220 220h160v-160H400v160Zm0 220h160v-160H400v160ZM180-400h160v-160H180v160Zm440 0h160v-160H620v160ZM180-180h160v-160H180v160Zm440 0h160v-160H620v160Z\"/>\n",
              "  </svg>\n",
              "    </button>\n",
              "\n",
              "  <style>\n",
              "    .colab-df-container {\n",
              "      display:flex;\n",
              "      gap: 12px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert {\n",
              "      background-color: #E8F0FE;\n",
              "      border: none;\n",
              "      border-radius: 50%;\n",
              "      cursor: pointer;\n",
              "      display: none;\n",
              "      fill: #1967D2;\n",
              "      height: 32px;\n",
              "      padding: 0 0 0 0;\n",
              "      width: 32px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert:hover {\n",
              "      background-color: #E2EBFA;\n",
              "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "      fill: #174EA6;\n",
              "    }\n",
              "\n",
              "    .colab-df-buttons div {\n",
              "      margin-bottom: 4px;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert {\n",
              "      background-color: #3B4455;\n",
              "      fill: #D2E3FC;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert:hover {\n",
              "      background-color: #434B5C;\n",
              "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
              "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
              "      fill: #FFFFFF;\n",
              "    }\n",
              "  </style>\n",
              "\n",
              "    <script>\n",
              "      const buttonEl =\n",
              "        document.querySelector('#df-317a4622-8f22-4ce0-a364-a0ff2ec822ca button.colab-df-convert');\n",
              "      buttonEl.style.display =\n",
              "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "\n",
              "      async function convertToInteractive(key) {\n",
              "        const element = document.querySelector('#df-317a4622-8f22-4ce0-a364-a0ff2ec822ca');\n",
              "        const dataTable =\n",
              "          await google.colab.kernel.invokeFunction('convertToInteractive',\n",
              "                                                    [key], {});\n",
              "        if (!dataTable) return;\n",
              "\n",
              "        const docLinkHtml = 'Like what you see? Visit the ' +\n",
              "          '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
              "          + ' to learn more about interactive tables.';\n",
              "        element.innerHTML = '';\n",
              "        dataTable['output_type'] = 'display_data';\n",
              "        await google.colab.output.renderOutput(dataTable, element);\n",
              "        const docLink = document.createElement('div');\n",
              "        docLink.innerHTML = docLinkHtml;\n",
              "        element.appendChild(docLink);\n",
              "      }\n",
              "    </script>\n",
              "  </div>\n",
              "\n",
              "\n",
              "<div id=\"df-ebb45ee7-956e-4d33-8f88-b586a3637d63\">\n",
              "  <button class=\"colab-df-quickchart\" onclick=\"quickchart('df-ebb45ee7-956e-4d33-8f88-b586a3637d63')\"\n",
              "            title=\"Suggest charts\"\n",
              "            style=\"display:none;\">\n",
              "\n",
              "<svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
              "     width=\"24px\">\n",
              "    <g>\n",
              "        <path d=\"M19 3H5c-1.1 0-2 .9-2 2v14c0 1.1.9 2 2 2h14c1.1 0 2-.9 2-2V5c0-1.1-.9-2-2-2zM9 17H7v-7h2v7zm4 0h-2V7h2v10zm4 0h-2v-4h2v4z\"/>\n",
              "    </g>\n",
              "</svg>\n",
              "  </button>\n",
              "\n",
              "<style>\n",
              "  .colab-df-quickchart {\n",
              "      --bg-color: #E8F0FE;\n",
              "      --fill-color: #1967D2;\n",
              "      --hover-bg-color: #E2EBFA;\n",
              "      --hover-fill-color: #174EA6;\n",
              "      --disabled-fill-color: #AAA;\n",
              "      --disabled-bg-color: #DDD;\n",
              "  }\n",
              "\n",
              "  [theme=dark] .colab-df-quickchart {\n",
              "      --bg-color: #3B4455;\n",
              "      --fill-color: #D2E3FC;\n",
              "      --hover-bg-color: #434B5C;\n",
              "      --hover-fill-color: #FFFFFF;\n",
              "      --disabled-bg-color: #3B4455;\n",
              "      --disabled-fill-color: #666;\n",
              "  }\n",
              "\n",
              "  .colab-df-quickchart {\n",
              "    background-color: var(--bg-color);\n",
              "    border: none;\n",
              "    border-radius: 50%;\n",
              "    cursor: pointer;\n",
              "    display: none;\n",
              "    fill: var(--fill-color);\n",
              "    height: 32px;\n",
              "    padding: 0;\n",
              "    width: 32px;\n",
              "  }\n",
              "\n",
              "  .colab-df-quickchart:hover {\n",
              "    background-color: var(--hover-bg-color);\n",
              "    box-shadow: 0 1px 2px rgba(60, 64, 67, 0.3), 0 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "    fill: var(--button-hover-fill-color);\n",
              "  }\n",
              "\n",
              "  .colab-df-quickchart-complete:disabled,\n",
              "  .colab-df-quickchart-complete:disabled:hover {\n",
              "    background-color: var(--disabled-bg-color);\n",
              "    fill: var(--disabled-fill-color);\n",
              "    box-shadow: none;\n",
              "  }\n",
              "\n",
              "  .colab-df-spinner {\n",
              "    border: 2px solid var(--fill-color);\n",
              "    border-color: transparent;\n",
              "    border-bottom-color: var(--fill-color);\n",
              "    animation:\n",
              "      spin 1s steps(1) infinite;\n",
              "  }\n",
              "\n",
              "  @keyframes spin {\n",
              "    0% {\n",
              "      border-color: transparent;\n",
              "      border-bottom-color: var(--fill-color);\n",
              "      border-left-color: var(--fill-color);\n",
              "    }\n",
              "    20% {\n",
              "      border-color: transparent;\n",
              "      border-left-color: var(--fill-color);\n",
              "      border-top-color: var(--fill-color);\n",
              "    }\n",
              "    30% {\n",
              "      border-color: transparent;\n",
              "      border-left-color: var(--fill-color);\n",
              "      border-top-color: var(--fill-color);\n",
              "      border-right-color: var(--fill-color);\n",
              "    }\n",
              "    40% {\n",
              "      border-color: transparent;\n",
              "      border-right-color: var(--fill-color);\n",
              "      border-top-color: var(--fill-color);\n",
              "    }\n",
              "    60% {\n",
              "      border-color: transparent;\n",
              "      border-right-color: var(--fill-color);\n",
              "    }\n",
              "    80% {\n",
              "      border-color: transparent;\n",
              "      border-right-color: var(--fill-color);\n",
              "      border-bottom-color: var(--fill-color);\n",
              "    }\n",
              "    90% {\n",
              "      border-color: transparent;\n",
              "      border-bottom-color: var(--fill-color);\n",
              "    }\n",
              "  }\n",
              "</style>\n",
              "\n",
              "  <script>\n",
              "    async function quickchart(key) {\n",
              "      const quickchartButtonEl =\n",
              "        document.querySelector('#' + key + ' button');\n",
              "      quickchartButtonEl.disabled = true;  // To prevent multiple clicks.\n",
              "      quickchartButtonEl.classList.add('colab-df-spinner');\n",
              "      try {\n",
              "        const charts = await google.colab.kernel.invokeFunction(\n",
              "            'suggestCharts', [key], {});\n",
              "      } catch (error) {\n",
              "        console.error('Error during call to suggestCharts:', error);\n",
              "      }\n",
              "      quickchartButtonEl.classList.remove('colab-df-spinner');\n",
              "      quickchartButtonEl.classList.add('colab-df-quickchart-complete');\n",
              "    }\n",
              "    (() => {\n",
              "      let quickchartButtonEl =\n",
              "        document.querySelector('#df-ebb45ee7-956e-4d33-8f88-b586a3637d63 button');\n",
              "      quickchartButtonEl.style.display =\n",
              "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "    })();\n",
              "  </script>\n",
              "</div>\n",
              "\n",
              "    </div>\n",
              "  </div>\n"
            ],
            "text/plain": [
              "   mice  efficacy  et  positively  appeared  investigations  preliminary  \\\n",
              "0     0         0   0           0         0               0            0   \n",
              "1     1         0   0           0         0               0            0   \n",
              "2     0         0   0           0         0               0            0   \n",
              "3     0         0   0           0         0               0            0   \n",
              "4     0         0   0           0         0               0            0   \n",
              "\n",
              "   pediatric  burden  reducing  ...  helpful  planning  valid  necessary  \\\n",
              "0          0       0         0  ...        0         0      0          0   \n",
              "1          0       0         0  ...        0         0      0          0   \n",
              "2          0       0         0  ...        1         0      0          0   \n",
              "3          0       0         0  ...        0         0      0          1   \n",
              "4          0       0         0  ...        0         0      0          0   \n",
              "\n",
              "   ninetyeight  independent  pressure  consider  monitoring  _label_  \n",
              "0            0            0         0         0           0        0  \n",
              "1            0            0         0         0           0        0  \n",
              "2            0            0         0         0           0        0  \n",
              "3            0            0         0         0           0        0  \n",
              "4            0            0         0         0           0        0  \n",
              "\n",
              "[5 rows x 403 columns]"
            ]
          },
          "execution_count": 27,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "dataset_feat_eng_train_selected.head()"
      ]
    },
    {
      "attachments": {},
      "cell_type": "markdown",
      "metadata": {
        "id": "ffIH343jLKRs"
      },
      "source": [
        "Parse out the Y labels from the dataset, and change the variable type to suit the models."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "ox0X7mDgLKg1"
      },
      "outputs": [],
      "source": [
        "x_features_train = dataset_feat_eng_train_selected.values[:, 0:-1]\n",
        "y_labels_train = dataset_feat_eng_train_selected.values[:, -1]\n",
        "\n",
        "x_features_test = dataset_feat_eng_test_selected.values[:, :-1]\n",
        "y_labels_test = dataset_feat_eng_test_selected.values[:, -1]"
      ]
    },
    {
      "attachments": {},
      "cell_type": "markdown",
      "metadata": {
        "id": "SDAccMSXDD7N"
      },
      "source": [
        "#### Iterate over ML models"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "lJyaQVzyDETH",
        "outputId": "f65fb64e-0d48-4457-c720-295f2a6d0c40"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "Random Forest: mean(accuracy)=0.815, std(accuracy)=0.012\n",
            "LASSO: mean(accuracy)=0.845, std(accuracy)=0.017\n",
            "KNN: mean(accuracy)=0.792, std(accuracy)=0.011\n",
            "Decision Tree: mean(accuracy)=0.797, std(accuracy)=0.013\n",
            "SVM: mean(accuracy)=0.746, std(accuracy)=0.001\n",
            "\n",
            "Best model is:\n",
            "LASSO\n"
          ]
        }
      ],
      "source": [
        "models = []\n",
        "models.append((\"Random Forest\", RandomForestClassifier(random_state=config_dict['seed'])))\n",
        "models.append((\"LASSO\", lm.LogisticRegression(solver='liblinear', penalty='l1', max_iter=1000, random_state=config_dict['seed'])))\n",
        "models.append((\"KNN\", KNeighborsClassifier()))\n",
        "models.append((\"Decision Tree\", DecisionTreeClassifier(random_state=config_dict['seed'])))\n",
        "models.append((\"SVM\", SVC(gamma='auto', random_state=config_dict['seed'])))\n",
        "\n",
        "results = []\n",
        "names = []\n",
        "best_mean_result = 0\n",
        "best_std_result = 0\n",
        "for name, model in models:\n",
        "  kfold = StratifiedKFold()\n",
        "  cv_results = cross_val_score(model, X=x_features_train, y=y_labels_train, scoring='accuracy', cv=kfold)\n",
        "  results.append(cv_results)\n",
        "  names.append(name)\n",
        "  print(name + \": mean(accuracy)=\" + str(round(np.mean(cv_results), 3)) + \", std(accuracy)=\" + str(round(np.std(cv_results), 3)))\n",
        "  if (best_mean_result < np.mean(cv_results)) or \\\n",
        "    ((best_mean_result == np.mean(cv_results)) and (best_std_result > np.std(cv_results))):\n",
        "    best_mean_result = np.mean(cv_results)\n",
        "    best_std_result = np.std(cv_results)\n",
        "    best_model_name = name\n",
        "    best_model = model\n",
        "print(\"\\nBest model is:\\n\" + best_model_name)"
      ]
    },
    {
      "attachments": {},
      "cell_type": "markdown",
      "metadata": {
        "id": "5HUN7jJE1IcP"
      },
      "source": [
        "Observe the distribution of the results across the validation folds:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 452
        },
        "id": "sx1tB-Vw1IuI",
        "outputId": "5da3c0da-f5e7-4163-ac9c-b797e63f634c"
      },
      "outputs": [
        {
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAiwAAAGzCAYAAAAMr0ziAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjcuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/bCgiHAAAACXBIWXMAAA9hAAAPYQGoP6dpAABR8ElEQVR4nO3deVyN6f8/8NepdNpDUSEdgxSaEEXZNTLIvpeyhRk742vX4EtjFpoxCN8sn4kPH7uxm0YY+9RgGpWdBpUG7Vk61+8Pv87H0eKcdHSX1/PxOA+6z3Vf9/s+6+tc57rvIxNCCBARERFJmF5ZF0BERET0NgwsREREJHkMLERERCR5DCxEREQkeQwsREREJHkMLERERCR5DCxEREQkeQwsREREJHkMLERERCR5DCwkOTKZDF9++aXW6925cwcymQwbN24s9ZqkSKFQYNiwYWVdBgAgKioKMpkMUVFRqmXDhg2DQqF4L9t/87bYuHEjZDIZfv/99/ey/fbt26N9+/bvZVsllZycjH79+sHKygoymQyhoaFlXRKRVhhYqFD5L/gymQy//fZbgeuFELC3t4dMJkP37t3LoMLS0759e8m88b+Lq1ev4ssvv8SdO3dKtL4Uwt677oMuSbk2TUyZMgVHjhzBrFmz8NNPP6FLly5lXRKRVgzKugCSNiMjI2zZsgWtW7dWW37ixAn8/fffkMvlZVQZvenq1atYsGAB2rdv/95GNoqzbt06KJVKrdYp6T4kJCRAT0+3n7+Kq+3o0aM63XZp+PXXX9GzZ0988cUXZV0KUYlwhIWK1bVrV2zfvh0vX75UW75lyxa4ubnB1ta2jCore0qlErm5uWVdhmRVqlRJp4FWCIGcnBwAgFwuR6VKlXS2rbcxNDSEoaFhmW1fEykpKahcuXJZl1HqcnNztQ7GVD4xsFCxBg8ejH/++QfHjh1TLXv+/Dl27NiBIUOGFLpOVlYWpk2bBnt7e8jlcjRo0ADffvst3vxh8GfPnmHKlCmoVq0azM3N0aNHD/z999+F9nn//n2MGDECNjY2kMvlaNSoEdavX//W+pOSkjB8+HDUqlULcrkcdnZ26NmzZ4mG9WUyGcaPH4/NmzejUaNGkMvlOHz4sFb1rVixAo0aNYKJiQmqVKmC5s2bY8uWLarri5r38eWXX0ImkxVZ28aNG9G/f38AQIcOHVRf5+XPKfn999/h4+MDa2trGBsbo06dOhgxYoTWtwEA/P333+jVqxdMTU1RvXp1TJkyBc+ePSvQrrB92bp1K9zc3GBubg4LCwu4uLjg+++/12gfFAoFunfvjiNHjqB58+YwNjbGmjVrVNcV9rVednY2xowZAysrK1hYWCAgIABPnjxRa1PUnKnX+3xbbYXNYUlJScHIkSNhY2MDIyMjuLq6YtOmTWpt8uddffvtt1i7di3q1q0LuVyOFi1a4OLFiwVqKsytW7fQv39/VK1aFSYmJmjZsiUOHDiguj7/610hBFauXKmqvTjffvstPD09YWVlBWNjY7i5uWHHjh2Fto2IiIC7u7vqMd22bdsCI06HDh1Cu3btVPd7ixYt1B73Rd1/b96u+XOltm7dirlz56JmzZowMTFBeno6Hj9+jC+++AIuLi4wMzODhYUFPv30U1y+fLlAv7m5ufjyyy/h6OgIIyMj2NnZoU+fPrh58yaEEFAoFOjZs2eh61laWmLMmDHF3n6kG/xKiIqlUCjQqlUr/Pvf/8ann34K4NWLT1paGgYNGoQffvhBrb0QAj169MDx48cxcuRINGnSBEeOHMH06dNx//59LF++XNV21KhRiIiIwJAhQ+Dp6Ylff/0V3bp1K1BDcnIyWrZsqQoM1apVw6FDhzBy5Eikp6dj8uTJRdbft29f/PXXX5gwYQIUCgVSUlJw7Ngx3Lt3r0Rfm/z666/4z3/+g/Hjx8Pa2hoKhULj+tatW4eJEyeiX79+mDRpEnJzc3HlyhWcP3++yPCnqbZt22LixIn44YcfMHv2bDg7OwMAnJ2dkZKSgs6dO6NatWqYOXMmKleujDt37mDXrl1abycnJwedOnXCvXv3MHHiRNSoUQM//fQTfv3117eue+zYMQwePBidOnXC0qVLAQBxcXE4ffo0Jk2aVOw+5EtISMDgwYMxZswYBAUFoUGDBsVuc/z48ahcuTK+/PJLJCQkYPXq1bh7967qjU9TmtT2upycHLRv3x43btzA+PHjUadOHWzfvh3Dhg3D06dPMWnSJLX2W7ZsQUZGBsaMGQOZTIavv/4affr0wa1bt4odOUpOToanpyeys7MxceJEWFlZYdOmTejRowd27NiB3r17o23btvjpp58wdOhQfPLJJwgICHjr/n7//ffo0aMH/Pz88Pz5c2zduhX9+/fH/v371Z6jCxYswJdffglPT08sXLgQhoaGOH/+PH799Vd07twZwKvANGLECDRq1AizZs1C5cqV8ccff+Dw4cMlftwvWrQIhoaG+OKLL/Ds2TMYGhri6tWr2LNnD/r37486deogOTkZa9asQbt27XD16lXUqFEDAJCXl4fu3bsjMjISgwYNwqRJk5CRkYFjx44hNjYWdevWhb+/P77++ms8fvwYVatWVW33559/Rnp6Ovz9/UtUN70jQVSIDRs2CADi4sWL4scffxTm5uYiOztbCCFE//79RYcOHYQQQjg4OIhu3bqp1tuzZ48AIP73f/9Xrb9+/foJmUwmbty4IYQQ4tKlSwKA+Pzzz9XaDRkyRAAQwcHBqmUjR44UdnZ2IjU1Va3toEGDhKWlpaqu27dvCwBiw4YNQgghnjx5IgCIb7755t1vECEEAKGnpyf++usvteWa1tezZ0/RqFGjYrcRGBgoHBwcCiwPDg4Wbz5dHRwcRGBgoOrv7du3CwDi+PHjau12796tui/fVWhoqAAg/vOf/6iWZWVliXr16hXY9pv7MmnSJGFhYSFevnxZZP9F7YMQr/YXgDh8+HCh171+W+Q/ft3c3MTz589Vy7/++msBQOzdu1e17M3HW1F9Fldbu3btRLt27VR/599OERERqmXPnz8XrVq1EmZmZiI9PV0I8d/HrJWVlXj8+LGq7d69ewUA8fPPPxfY1usmT54sAIhTp06plmVkZIg6deoIhUIh8vLy1PZz3LhxxfaXL/8x+3rtjRs3Fh07dlQtu379utDT0xO9e/dW244QQiiVSiGEEE+fPhXm5ubCw8ND5OTkFNpGiIK3db43b9fjx48LAOKjjz4qUGNubm6BOm7fvi3kcrlYuHChatn69esFALFs2bIC28uvKSEhQQAQq1evVru+R48eQqFQqNVO7w+/EqK3GjBgAHJycrB//35kZGRg//79RX4yOnjwIPT19TFx4kS15dOmTYMQAocOHVK1A1Cg3ZujJUII7Ny5E76+vhBCIDU1VXXx8fFBWloaYmJiCq3F2NgYhoaGiIqKKvA1QEm1a9cODRs2LFF9lStXxt9//63xUH9pyZ+3sH//frx48eKd+jp48CDs7OzQr18/1TITExOMHj1aozqysrLUvl7UVp06deDj46Nx+9GjR6uNUHz22WcwMDBQPf505eDBg7C1tcXgwYNVyypVqoSJEyciMzMTJ06cUGs/cOBAVKlSRfV3mzZtALz6uudt23F3d1ebFG9mZobRo0fjzp07uHr1aonqNzY2Vv3/yZMnSEtLQ5s2bdSea3v27IFSqcT8+fMLTHjOH706duwYMjIyMHPmTBgZGRXapiQCAwPVagRezWPKryMvLw///PMPzMzM0KBBA7W6d+7cCWtra0yYMKFAv/k1OTo6wsPDA5s3b1Zd9/jxYxw6dAh+fn7vVDuVHAMLvVW1atXg7e2NLVu2YNeuXcjLy1N7w3rd3bt3UaNGDZibm6stzx86v3v3rupfPT091K1bV63dm0P8jx49wtOnT7F27VpUq1ZN7TJ8+HAAr+YKFEYul2Pp0qU4dOgQbGxs0LZtW3z99ddISkrS/kb4/+rUqVPi+mbMmAEzMzO4u7ujfv36GDduHE6fPl3iWjTVrl079O3bFwsWLIC1tTV69uyJDRs2FDrv5G3u3r2LevXqFXjBfttXMwDw+eefw9HREZ9++ilq1aqFESNGqOYAaerN2/9t6tevr/a3mZkZ7OzsdH5o8t27d1G/fv0Cb+RvPg/y1a5dW+3v/PDytqB99+7dQm/7orajqf3796Nly5YwMjJC1apVUa1aNaxevRppaWmqNjdv3oSenp5agH/TzZs3AQCNGzcuUR1FKexxoFQqsXz5ctSvXx9yuRzW1taoVq0arly5UqDuBg0awMCg+BkRAQEBOH36tOo23L59O168eIGhQ4eW6r6Q5hhYSCNDhgzBoUOHEBYWhk8//fS9HW2QP/vf398fx44dK/Ti5eVV5PqTJ0/GtWvXEBISAiMjI8ybNw/Ozs74448/SlTPm5/qtKnP2dkZCQkJ2Lp1K1q3bo2dO3eidevWCA4OVvVX1Ce3vLy8EtWb3+eOHTtw9uxZjB8/XjVB2M3NDZmZmSXuV1vVq1fHpUuXsG/fPtU8p08//RSBgYEa9/Hm7a9L73Kba0tfX7/Q5eKNiervw6lTp9CjRw8YGRlh1apVOHjwII4dO4YhQ4borB5tH/eFPQ6WLFmCqVOnom3btoiIiMCRI0dw7NgxNGrUqERHEQ0aNAiVKlVSjbJERESgefPmGoVz0g1OuiWN9O7dG2PGjMG5c+ewbdu2Its5ODjgl19+QUZGhtooS3x8vOr6/H+VSqXq006+hIQEtf7yjyDKy8uDt7d3iWqvW7cupk2bhmnTpuH69eto0qQJvvvuO0RERJSov3epz9TUFAMHDsTAgQPx/Plz9OnTB4sXL8asWbNgZGSEKlWq4OnTpwXW0+ST8tuGqVu2bImWLVti8eLF2LJlC/z8/LB161aMGjXqrX3nc3BwQGxsLIQQatt7834riqGhIXx9feHr6wulUonPP/8ca9aswbx58woduXlX169fR4cOHVR/Z2Zm4uHDh+jatatqWWG3+fPnz/Hw4UO1ZdrU5uDggCtXrkCpVKqNsrz5PHhXDg4Ohd7277KdnTt3wsjICEeOHFE7LH3Dhg1q7erWrQulUomrV6+iSZMmhfaVP4IaGxuLevXqFbnN4h73H330kUZ179ixAx06dEB4eLja8qdPn8La2lqtpvPnz+PFixfFTmiuWrUqunXrhs2bN8PPzw+nT5/m2YHLGEdYSCNmZmZYvXo1vvzyS/j6+hbZrmvXrsjLy8OPP/6otnz58uWQyWSqI43y/33zKKM3XxD09fXRt29f7Ny5E7GxsQW29+jRoyJryc7OLnCelLp168Lc3LxEX4cURpv6/vnnH7XrDA0N0bBhQwghVHNL6tati7S0NFy5ckXV7uHDh9i9e/dbazE1NQWAAi/8T548KfDJOP8NRtvboWvXrnjw4IHaIa7Z2dlYu3btW9d9c//19PTw8ccfq9VR1D6U1Nq1a9Xm7axevRovX75UPf6AV7f5yZMnC6z35qd7bWrr2rUrkpKS1ML9y5cvsWLFCpiZmaFdu3Yl2Z1Ct3PhwgWcPXtWtSwrKwtr166FQqEo9uuaoujr60Mmk6nt/507d7Bnzx61dr169YKenh4WLlxYYAQj//HWuXNnmJubIyQkpMBz8fXHZN26dXHu3Dk8f/5ctWz//v1ITEzUqu43H+fbt2/H/fv31Zb17dsXqampBV6j3qwJAIYOHYqrV69i+vTp0NfXx6BBgzSuh0ofR1hIY5oM3fv6+qJDhw6YM2cO7ty5A1dXVxw9ehR79+7F5MmTVZ+4mjRpgsGDB2PVqlVIS0uDp6cnIiMjcePGjQJ9fvXVVzh+/Dg8PDwQFBSEhg0b4vHjx4iJicEvv/yCx48fF1rLtWvX0KlTJwwYMAANGzaEgYEBdu/ejeTk5FJ94dG0vs6dO8PW1hZeXl6wsbFBXFwcfvzxR3Tr1k01GjVo0CDMmDEDvXv3xsSJE5GdnY3Vq1fD0dGxyMnF+Zo0aQJ9fX0sXboUaWlpkMvl6NixI7Zs2YJVq1ahd+/eqFu3LjIyMrBu3TpYWFiojTRoIigoCD/++CMCAgIQHR0NOzs7/PTTTzAxMXnruqNGjcLjx4/RsWNH1KpVC3fv3sWKFSvQpEkT1ZyLovahevXqWtWZ7/nz56rHQEJCAlatWoXWrVujR48eanWNHTsWffv2xSeffILLly/jyJEjap/Kta1t9OjRWLNmDYYNG4bo6GgoFArs2LFD9Sn9zTleJTVz5kzVKQcmTpyIqlWrYtOmTbh9+zZ27txZorP/duvWDcuWLUOXLl0wZMgQpKSkYOXKlahXr55akK5Xrx7mzJmDRYsWoU2bNujTpw/kcjkuXryIGjVqICQkBBYWFli+fDlGjRqFFi1aYMiQIahSpQouX76M7Oxs1XlpRo0ahR07dqBLly4YMGAAbt68iYiIiAJz3IrTvXt3LFy4EMOHD4enpyf+/PNPbN68ucAITUBAAP71r39h6tSpuHDhAtq0aYOsrCz88ssv+Pzzz9XOv9KtWzdYWVlh+/bt+PTTT0v8OKRSUhaHJpH0vX5Yc3HePKxZiFeHVU6ZMkXUqFFDVKpUSdSvX1988803BQ4FzMnJERMnThRWVlbC1NRU+Pr6isTExEIPM01OThbjxo0T9vb2olKlSsLW1lZ06tRJrF27VtXmzcOaU1NTxbhx44STk5MwNTUVlpaWwsPDQ+2QXG2gmMNCNalvzZo1om3btsLKykrI5XJRt25dMX36dJGWlqbW19GjR0Xjxo2FoaGhaNCggYiIiNDosGYhhFi3bp346KOPhL6+vuoQ3JiYGDF48GBRu3ZtIZfLRfXq1UX37t3F77//XqLb4e7du6JHjx7CxMREWFtbi0mTJonDhw+/9bDmHTt2iM6dO4vq1asLQ0NDUbt2bTFmzBjx8OHDt+5D/v6++Vgr6rbIf/yeOHFCjB49WlSpUkWYmZkJPz8/8c8//6itm5eXJ2bMmCGsra2FiYmJ8PHxETdu3ND49hWi4OG3Qrx6TAwfPlxYW1sLQ0ND4eLionps5st/zBZ26H1hz4PC3Lx5U/Tr109UrlxZGBkZCXd3d7F///5C+9P0sObw8HBRv359IZfLhZOTk9iwYUOhj0EhXh0m3LRpUyGXy0WVKlVEu3btxLFjx9Ta7Nu3T3h6egpjY2NhYWEh3N3dxb///W+1Nt99952oWbOmkMvlwsvLS/z+++9FHta8ffv2AnXk5uaKadOmCTs7O2FsbCy8vLzE2bNnC71vsrOzxZw5c0SdOnVUz9d+/fqJmzdvFuj3888/FwDEli1bNLrtSHdkQpTBrC4iIqJyYMqUKQgPD0dSUpJGI4mkO5zDQkREVIjc3FxERESgb9++DCsSwDksREREr0lJScEvv/yCHTt24J9//inwUwpUNhhYiIiIXnP16lX4+fmhevXq+OGHH4o8bJveL85hISIiIsnjHBYiIiKSPAYWIiIikrwKM4dFqVTiwYMHMDc35y9pEhERlRNCCGRkZKBGjRrFnuywwgSWBw8ewN7evqzLICIiohJITExErVq1iry+wgSW/FNdJyYmwsLCooyrISIiIk2kp6fD3t7+rT9ZUWECS/7XQBYWFgwsRERE5czbpnNw0i0RERFJHgMLERERSR4DCxEREUkeAwsRERFJHgMLERERSR4DCxEREUkeAwsRERFJHgMLERERSR4DCxEREUkeAwsRERFJHgMLERERSR4DCxEREUlehfnxQ6LiZGdnIz4+XuP2OTk5uHPnDhQKBYyNjbXalpOTE0xMTLQtkYiIisHAQh+E+Ph4uLm5vZdtRUdHo1mzZu9lW0REHwoGFvogODk5ITo6WuP2cXFx8Pf3R0REBJydnbXeFhERlS4GFvogmJiYlGjUw9nZmaMlREQSwEm3REREJHkMLERERCR5DCxEREQkeQwsREREJHkMLERERCR5DCxEREQkeQwsREREJHkMLERERCR5DCxEREQkeQwsREREJHkMLERERCR5DCxEREQkeQwsREREJHkMLERERCR5DCxEREQkeQwsREREJHkMLERERCR5DCxEREQkeQwsREREJHkMLERERCR5DCxEREQkeQwsREREJHkMLERERCR5DCxEREQkeQwsREREJHkMLERERCR5DCxEREQkeQwsREREJHklCiwrV66EQqGAkZERPDw8cOHChWLbh4aGokGDBjA2Noa9vT2mTJmC3NxctTb379+Hv78/rKysYGxsDBcXF/z+++8lKY+IiIgqGANtV9i2bRumTp2KsLAweHh4IDQ0FD4+PkhISED16tULtN+yZQtmzpyJ9evXw9PTE9euXcOwYcMgk8mwbNkyAMCTJ0/g5eWFDh064NChQ6hWrRquX7+OKlWqvPseEhERUbmndWBZtmwZgoKCMHz4cABAWFgYDhw4gPXr12PmzJkF2p85cwZeXl4YMmQIAEChUGDw4ME4f/68qs3SpUthb2+PDRs2qJbVqVNH650hIiKiikmrr4SeP3+O6OhoeHt7/7cDPT14e3vj7Nmzha7j6emJ6Oho1ddGt27dwsGDB9G1a1dVm3379qF58+bo378/qlevjqZNm2LdunXF1vLs2TOkp6erXYiIiKhi0iqwpKamIi8vDzY2NmrLbWxskJSUVOg6Q4YMwcKFC9G6dWtUqlQJdevWRfv27TF79mxVm1u3bmH16tWoX78+jhw5gs8++wwTJ07Epk2biqwlJCQElpaWqou9vb02u0JERETliM6PEoqKisKSJUuwatUqxMTEYNeuXThw4AAWLVqkaqNUKtGsWTMsWbIETZs2xejRoxEUFISwsLAi+501axbS0tJUl8TERF3vChEREZURreawWFtbQ19fH8nJyWrLk5OTYWtrW+g68+bNw9ChQzFq1CgAgIuLC7KysjB69GjMmTMHenp6sLOzQ8OGDdXWc3Z2xs6dO4usRS6XQy6Xa1M+ERERlVNajbAYGhrCzc0NkZGRqmVKpRKRkZFo1apVoetkZ2dDT099M/r6+gAAIQQAwMvLCwkJCWptrl27BgcHB23KIyIiogpK66OEpk6disDAQDRv3hzu7u4IDQ1FVlaW6qihgIAA1KxZEyEhIQAAX19fLFu2DE2bNoWHhwdu3LiBefPmwdfXVxVcpkyZAk9PTyxZsgQDBgzAhQsXsHbtWqxdu7YUd5WIiIjKK60Dy8CBA/Ho0SPMnz8fSUlJaNKkCQ4fPqyaiHvv3j21EZW5c+dCJpNh7ty5uH//PqpVqwZfX18sXrxY1aZFixbYvXs3Zs2ahYULF6JOnToIDQ2Fn59fKewiERERlXcykf+9TDmXnp4OS0tLpKWlwcLCoqzLoXIuJiYGbm5uiI6ORrNmzcq6HCKiCkvT92/+lhARERFJHgMLERERSR4DCxEREUkeAwsRERFJHgMLERERSR4DCxEREUkeAwsRERFJHgMLERERSR4DCxEREUkeAwsRERFJHgMLERERSR4DCxEREUme1r/WTCQV169fR0ZGhk76jouLU/tXV8zNzVG/fn2dboOIqCJgYKFy6fr163B0dNT5dvz9/XW+jWvXrjG0EBG9BQMLlUv5IysRERFwdnYu9f5zcnJw584dKBQKGBsbl3r/wKvRG39/f52NEhERVSQMLFSuOTs7o1mzZjrp28vLSyf9EhGR9jjploiIiCSPgYWIiIgkj4GFiIiIJI+BhYiIiCSPgYWIiIgkj4GFiIiIJI+BhYiIiCSPgYWIiIgkj4GFiIiIJI+BhYiIiCSPgYWIiIgkj4GFiIiIJI+BhYiIiCSPgYWIiIgkj4GFiIiIJI+BhYiIiCTPoKwLICoJ2ctcNLXVg/HTa8CD8pm7jZ9eQ1NbPche5pZ1KUREksfAQuWSUeY9xIwxA06OAU6WdTUl4wwgZowZ4jLvAfAs63KIiCSNgYXKpVyz2mi2JhObN2+Gs5NTWZdTInHx8fDz80N419plXQoRkeQxsFC5JAyM8EeSEjmVHYEaTcq6nBLJSVLijyQlhIFRWZdCRCR55fPLfyIiIvqgMLAQERGR5DGwEBERkeQxsBAREZHkMbAQERGR5JUosKxcuRIKhQJGRkbw8PDAhQsXim0fGhqKBg0awNjYGPb29pgyZQpycws/WdZXX30FmUyGyZMnl6Q0IiIiqoC0Dizbtm3D1KlTERwcjJiYGLi6usLHxwcpKSmFtt+yZQtmzpyJ4OBgxMXFITw8HNu2bcPs2bMLtL148SLWrFmDjz/+WPs9ISIiogpL68CybNkyBAUFYfjw4WjYsCHCwsJgYmKC9evXF9r+zJkz8PLywpAhQ6BQKNC5c2cMHjy4wKhMZmYm/Pz8sG7dOlSpUqVke0NEREQVklaB5fnz54iOjoa3t/d/O9DTg7e3N86ePVvoOp6enoiOjlYFlFu3buHgwYPo2rWrWrtx48ahW7duan0X59mzZ0hPT1e7EBERUcWk1ZluU1NTkZeXBxsbG7XlNjY2iI+PL3SdIUOGIDU1Fa1bt4YQAi9fvsTYsWPVvhLaunUrYmJicPHiRY1rCQkJwYIFC7Qpn4iIiMopnR8lFBUVhSVLlmDVqlWIiYnBrl27cODAASxatAgAkJiYiEmTJmHz5s0wMtL8FOWzZs1CWlqa6pKYmKirXSAiIqIyptUIi7W1NfT19ZGcnKy2PDk5Gba2toWuM2/ePAwdOhSjRo0CALi4uCArKwujR4/GnDlzEB0djZSUFDRr1ky1Tl5eHk6ePIkff/wRz549g76+foF+5XI55HK5NuUTERFROaXVCIuhoSHc3NwQGRmpWqZUKhEZGYlWrVoVuk52djb09NQ3kx9AhBDo1KkT/vzzT1y6dEl1ad68Ofz8/HDp0qVCwwoRERF9WLT+teapU6ciMDAQzZs3h7u7O0JDQ5GVlYXhw4cDAAICAlCzZk2EhIQAAHx9fbFs2TI0bdoUHh4euHHjBubNmwdfX1/o6+vD3NwcjRs3VtuGqakprKysCiwnIiKiD5PWgWXgwIF49OgR5s+fj6SkJDRp0gSHDx9WTcS9d++e2ojK3LlzIZPJMHfuXNy/fx/VqlWDr68vFi9eXHp7QURERBWaTAghyrqI0pCeng5LS0ukpaXBwsKirMshHYuJiYGbmxuio6PV5j+VJxVhH4iI3pWm79/8LSEiIiKSPAYWIiIikjwGFiIiIpI8BhYiIiKSPK2PEiLNZWdnF/mTBYXJycnBnTt3oFAoYGxsrNW2nJycYGJiom2JRERE5QIDiw7Fx8fDzc3tvWyLR5oQEVFFxsCiQ05OToiOjta4fVxcHPz9/REREQFnZ2ett0VERFRRMbDokImJSYlGPZydnTlaQkRE9BpOuiUiIiLJY2AhIiIiyWNgISIiIsljYCEiIiLJY2AhIiIiyWNgISIiIsljYCEiIiLJY2AhIiIiyWNgISIiIsljYCEiIiLJY2AhIiIiyWNgISIiIsnjjx9SuZSdnQ0AiImJ0Un/OTk5uHPnDhQKBYyNjXWyjbi4OJ30S0RUETGwULkUHx8PAAgKCirjSt6dubl5WZdARCR5DCxULvXq1QsA4OTkBBMTk1LvPy4uDv7+/oiIiICzs3Op95/P3Nwc9evX11n/REQVBQMLlUvW1tYYNWqUzrfj7OyMZs2a6Xw7RERUPE66JSIiIsljYCEiIiLJY2AhIiIiyWNgISIiIsnjpFv6IGRnZ6sOhdZE/jlSSnKuFF0duURE9CFjYKEPQnx8PNzc3LRez9/fX+t1oqOjeWQREVEpY2ChD4KTkxOio6M1bv8uZ7p1cnLStjwiInoLBhb6IJiYmGg96uHl5aWjaoiISFucdEtERESSx8BCREREksevhIiISCN5eXk4deoUHj58CDs7O7Rp0wb6+vplXRZ9IDjCQkREb7Vr1y7Uq1cPHTp0wJAhQ9ChQwfUq1cPu3btKuvS6APBwEJERMXatWsX+vXrBxcXF5w9exYZGRk4e/YsXFxc0K9fP4YWei9kQghR1kWUhvT0dFhaWiItLQ0WFhZlXU6JxMTEwM3NjefxICLJyMvLQ7169eDi4oI9e/ZAT++/n3OVSiV69eqF2NhYXL9+nV8PUYlo+v7NERYiIirSqVOncOfOHcyePVstrACAnp4eZs2ahdu3b+PUqVNlVCF9KBhYiIioSA8fPgQANG7cuNDr85fntyPSFR4lpIXr168jIyNDZ/2/y+/XaMPc3Bz169fX6TaIqGKws7MDAMTGxqJly5YFro+NjVVrR6QrnMOioevXr8PR0bHU+y0r165dY2ghorfiHBbSNU3fv0s0wrJy5Up88803SEpKgqurK1asWAF3d/ci24eGhmL16tW4d+8erK2t0a9fP4SEhMDIyAgAEBISgl27diE+Ph7Gxsbw9PTE0qVL0aBBg5KUpxP5IysRERFwdnbWyTbe5fdrNBUXFwd/f3+djhQRUcWhr6+P7777Dv369UOvXr0wa9YsNG7cGLGxsQgJCcH+/fuxY8cOhhXSPaGlrVu3CkNDQ7F+/Xrx119/iaCgIFG5cmWRnJxcaPvNmzcLuVwuNm/eLG7fvi2OHDki7OzsxJQpU1RtfHx8xIYNG0RsbKy4dOmS6Nq1q6hdu7bIzMzUuK60tDQBQKSlpWm7SxqJjo4WAER0dLRO+n9fKsp+ENH7tXPnTqFQKAQA1aVOnTpi586dZV0alXOavn9rPcKybNkyBAUFYfjw4QCAsLAwHDhwAOvXr8fMmTMLtD9z5gy8vLwwZMgQAIBCocDgwYNx/vx5VZvDhw+rrbNx40ZUr14d0dHRaNu2rbYlEhFRKevTpw969uzJM91SmdEqsDx//hzR0dGYNWuWapmenh68vb1x9uzZQtfx9PREREQELly4AHd3d9y6dQsHDx7E0KFDi9xOWloaAKBq1apFtnn27BmePXum+js9PV2bXSEiIi3p6+ujffv2ZV0GfaC0CiypqanIy8uDjY2N2nIbGxvEx8cXus6QIUOQmpqK1q1bQwiBly9fYuzYsZg9e3ah7ZVKJSZPngwvL68iD6MDXs17WbBggTblExERUTml8/OwREVFYcmSJVi1ahViYmKwa9cuHDhwAIsWLSq0/bhx4xAbG4utW7cW2++sWbOQlpamuiQmJuqifCIiIpIArUZYrK2toa+vj+TkZLXlycnJsLW1LXSdefPmYejQoRg1ahQAwMXFBVlZWRg9ejTmzJmjdojc+PHjsX//fpw8eRK1atUqtha5XA65XK5N+URERFROaTXCYmhoCDc3N0RGRqqWKZVKREZGolWrVoWuk52dXeB0zvmTtMT/PwWMEALjx4/H7t278euvv6JOnTpa7QQRERFVbFofJTR16lQEBgaiefPmcHd3R2hoKLKyslRHDQUEBKBmzZoICQkBAPj6+mLZsmVo2rQpPDw8cOPGDcybNw++vr6q4DJu3Dhs2bIFe/fuhbm5OZKSkgAAlpaWOjsfCREREZUfWgeWgQMH4tGjR5g/fz6SkpLQpEkTHD58WDUR9969e2ojKnPnzoVMJsPcuXNx//59VKtWDb6+vli8eLGqzerVqwGgwOzzDRs2YNiwYSXYLSIiIqpIeGp+DcXExMDNzQ3R0dFo1qxZqff/vlSU/SAioopB0/dv/lozERERSR4DCxEREUkeAwsRERFJHgMLERERSR4DCxEREUkeAwsRERFJHgMLERERSR4DCxEREUkeAwsRERFJHgMLERERSR4DCxEREUkeAwsRERFJHgMLERERSR4DCxEREUkeAwsRERFJnkFZF1BeyF7moqmtHoyfXgMelN+cZ/z0Gpra6kH2MresSyEiItIYA4uGjDLvIWaMGXByDHCyrKspOWcAMWPMEJd5D4BnWZdDRESkEQYWDeWa1UazNZnYvHkznJ2cyrqcEouLj4efnx/Cu9Yu61KIiIg0xsCiIWFghD+SlMip7AjUaFLW5ZRYTpISfyQpIQyMyroUIiIijTGwENF7lZ2djfj4eK3WycnJwZ07d6BQKGBsbKzxek5OTjAxMdG2RCKSIAYWInqv4uPj4ebm9l62FR0djWbNmr2XbRGRbjGwENF75eTkhOjoaK3WiYuLg7+/PyIiIuDs7KzVtoioYmBgIaL3ysTEpMSjHs7OzhwxIfpAld8TihAREdEHg4GFiIiIJI+BhYiIiCSPgYWIiIgkj4GFiIiIJI+BhYiIiCSPgYWIiIgkj+dh0VB2djYAICYmRmfbKOnpx7URFxenk36JiIh0iYFFQ/m/fRIUFFTGlZQOc3Pzsi6BiIhIYwwsGurVqxcA3f6YWklPP64tc3Nz1K9fX2f9ExERlTYGFg1ZW1tj1KhR72VbPP04ERGROk66JSIiIsljYCEiIiLJY2AhIiIiyWNgISIiIsljYCEiIiLJ41FCOpSdna06f4sm8k/qVpKTu+nycGsiIqKyxsCiQ/Hx8XBzc9N6PX9/f63XiY6O5qHQRERUYZUosKxcuRLffPMNkpKS4OrqihUrVsDd3b3I9qGhoVi9ejXu3bsHa2tr9OvXDyEhITAyMipxn+WBk5MToqOjNW7/Lqfmd3Jy0rY8IiKickPrwLJt2zZMnToVYWFh8PDwQGhoKHx8fJCQkIDq1asXaL9lyxbMnDkT69evh6enJ65du4Zhw4ZBJpNh2bJlJeqzvDAxMdF61MPLy0tH1RAREZVfWk+6XbZsGYKCgjB8+HA0bNgQYWFhMDExwfr16wttf+bMGXh5eWHIkCFQKBTo3LkzBg8ejAsXLpS4TyIiIvqwaBVYnj9/jujoaHh7e/+3Az09eHt74+zZs4Wu4+npiejoaFVAuXXrFg4ePIiuXbuWuE8AePbsGdLT09UuREREVDFp9ZVQamoq8vLyYGNjo7bcxsamyKNhhgwZgtTUVLRu3RpCCLx8+RJjx47F7NmzS9wnAISEhGDBggXalE9ERETllM6PEoqKisKSJUuwatUqeHh44MaNG5g0aRIWLVqEefPmlbjfWbNmYerUqaq/09PTYW9vXxolE5GWrl+/joyMDJ31/y6H/GuKv2JOJG1aBRZra2vo6+sjOTlZbXlycjJsbW0LXWfevHkYOnSo6peOXVxckJWVhdGjR2POnDkl6hMA5HI55HK5NuUTkQ5cv34djo6O72VbJTnkXxvXrl1jaCGSKK0Ci6GhIdzc3BAZGYlevXoBAJRKJSIjIzF+/PhC18nOzoaenvpUGX19fQCAEKJEfRKRdOSPrERERMDZ2Vkn23iXQ/41ERcXB39/f52OEhHRu9H6K6GpU6ciMDAQzZs3h7u7O0JDQ5GVlYXhw4cDAAICAlCzZk2EhIQAAHx9fbFs2TI0bdpU9ZXQvHnz4Ovrqwoub+uTiKTP2dlZpycv5CH/RB82rQPLwIED8ejRI8yfPx9JSUlo0qQJDh8+rJo0e+/ePbURlblz50Imk2Hu3Lm4f/8+qlWrBl9fXyxevFjjPomIiOjDJhNCiLIuojSkp6fD0tISaWlpsLCwKOtyiD4YMTExcHNzK9c/D1ER9oGovNL0/Zu/1kxERESSx8BCREREksfAQkRERJLHwEJERESSx8BCREREkqfzU/MTEdH7o83PJOSfkO990Pakf/ypBHoTAwsRUQXxPn8m4X3gTyXQ6xhYiIgqCG1/JkGqIyz8qQQqDAMLEVEFIXuZi6a2emhmpw9nW02mKJrCq04jndelLeOn+mhqqwfZy9yyLoUkhIGFiKiCMMq8h5gxZsDJMcDJsq6m5JwBxIwxQ1zmPQCeZV0OSQQDCxFRBZFrVhvN1mRi8+bNcHZyKutySiwuPh5+fn4I71q7rEshCWFgISKqIISBEf5IUiKnsiNQo0lZl1NiOUlK/JGkhDAwKutSSEJ4HhYiIiKSPAYWIiIikjwGFiIiIpI8BhYiIiKSPAYWIiIikjwGFiIiIpI8BhYiIiKSPAYWIiIikjwGFiIiIpI8BhYiIiKSPAYWIiIikjz+lhARvRPZy1w0tdWD8dNrwIPy+RnI+Ok1NLXVg+xlblmXQkRFYGAhondilHkPMWPMgJNjgJNlXU3JOAOIGWOGuMx7ADzLuhwiKgQDCxG9k1yz2mi2JhObN2+Gs5NTWZdTInHx8fDz80N419plXco7yc7OBgDExMTobBs5OTm4c+cOFAoFjI2NdbKNuLg4nfRL5RsDCxG9E2FghD+SlMip7AjUaFLW5ZRITpISfyQpIQyMyrqUdxIfHw8ACAoKKuNKSoe5uXlZl0ASwsBCRFRB9OrVCwDg5OQEExOTt7aPi4uDv7+/jqt6JSIiAs7Ozhq3Nzc3R/369XVYEZU3DCxERBWEtbU1Ro0apXF7JycnREdHa7WNkn4lpGmIIioKAwsR0QfKxMQEzZo103o9Ly8vHVRDVLzyeQwiERERfVAYWIiIiEjyGFiIiIhI8hhYiIiISPI46ZaI3klFOFkZT1RGJH0MLET0TirSycp4ojIi6WJgIaJ3ou3Jykoi/wRn2p58TBs8URmRtDGwENE70fZkZe/C2dm5ROcNIaLyj5NuiYiISPIYWIiIiEjyGFiIiIhI8hhYiIiISPJKFFhWrlwJhUIBIyMjeHh44MKFC0W2bd++PWQyWYFLt27dVG0yMzMxfvx41KpVC8bGxmjYsCHCwsJKUhoRERFVQFoHlm3btmHq1KkIDg5GTEwMXF1d4ePjg5SUlELb79q1Cw8fPlRdYmNjoa+vj/79+6vaTJ06FYcPH0ZERATi4uIwefJkjB8/Hvv27Sv5nhEREVGFoXVgWbZsGYKCgjB8+HDVSIiJiQnWr19faPuqVavC1tZWdTl27BhMTEzUAsuZM2cQGBiI9u3bQ6FQYPTo0XB1dS125ObZs2dIT09XuxAREVHFpFVgef78OaKjo+Ht7f3fDvT04O3tjbNnz2rUR3h4OAYNGgRTU1PVMk9PT+zbtw/379+HEALHjx/HtWvX0Llz5yL7CQkJgaWlpepib2+vza4QERFROaJVYElNTUVeXh5sbGzUltvY2CApKemt61+4cAGxsbEFTjK1YsUKNGzYELVq1YKhoSG6dOmClStXom3btkX2NWvWLKSlpakuiYmJ2uwKERERlSPv9Uy34eHhcHFxgbu7u9ryFStW4Ny5c9i3bx8cHBxw8uRJjBs3DjVq1FAbzXmdXC6HXC5/H2UTERFRGdMqsFhbW0NfXx/Jyclqy5OTk2Fra1vsullZWdi6dSsWLlyotjwnJwezZ8/G7t27VUcOffzxx7h06RK+/fbbIgMLERERfTi0+krI0NAQbm5uiIyMVC1TKpWIjIxEq1atil13+/btePbsGfz9/dWWv3jxAi9evICennop+vr6UCqV2pRHREREFZTWXwlNnToVgYGBaN68Odzd3REaGoqsrCwMHz4cABAQEICaNWsiJCREbb3w8HD06tULVlZWasstLCzQrl07TJ8+HcbGxnBwcMCJEyfwr3/9C8uWLXuHXSMiKcrOzkZ8fLxW68TFxan9qyld/oI0Eb1fWgeWgQMH4tGjR5g/fz6SkpLQpEkTHD58WDUR9969ewVGSxISEvDbb7/h6NGjhfa5detWzJo1C35+fnj8+DEcHBywePFijB07tgS7RERSFh8fDzc3txKt++YI7dtER0fz152JKgiZEEKUdRGlIT09HZaWlkhLS4OFhUVZl0NERSjJCEtOTg7u3LkDhUIBY2NjjdfjCAuR9Gn6/s3AQkRERGVG0/dv/vghERERSR4DCxEREUkeAwsRERFJHgMLERERSR4DCxEREUkeAwsRERFJHgMLERERSR4DCxEREUkeAwsRERFJHgMLERERSR4DCxEREUkeAwsRERFJHgMLERERSR4DCxEREUkeAwsRERFJHgMLERERSR4DCxEREUkeAwsRERFJHgMLERERSR4DCxEREUkeAwsRERFJHgMLERERSR4DCxEREUkeAwsRERFJHgMLERERSR4DCxEREUkeAwsRERFJHgMLERERSR4DCxEREUkeAwsRERFJHgMLERERSR4DCxEREUkeAwsRERFJHgMLERERSR4DCxEREUkeAwsRERFJHgMLERERSR4DCxEREUkeAwsRERFJHgMLERERSV6JAsvKlSuhUChgZGQEDw8PXLhwoci27du3h0wmK3Dp1q2bWru4uDj06NEDlpaWMDU1RYsWLXDv3r2SlEdEREQVjNaBZdu2bZg6dSqCg4MRExMDV1dX+Pj4ICUlpdD2u3btwsOHD1WX2NhY6Ovro3///qo2N2/eROvWreHk5ISoqChcuXIF8+bNg5GRUcn3jIiIiCoMmRBCaLOCh4cHWrRogR9//BEAoFQqYW9vjwkTJmDmzJlvXT80NBTz58/Hw4cPYWpqCgAYNGgQKlWqhJ9++qkEu/BKeno6LC0tkZaWBgsLixL3Q0RERO+Ppu/fWo2wPH/+HNHR0fD29v5vB3p68Pb2xtmzZzXqIzw8HIMGDVKFFaVSiQMHDsDR0RE+Pj6oXr06PDw8sGfPnmL7efbsGdLT09UuREREVDFpFVhSU1ORl5cHGxsbteU2NjZISkp66/oXLlxAbGwsRo0apVqWkpKCzMxMfPXVV+jSpQuOHj2K3r17o0+fPjhx4kSRfYWEhMDS0lJ1sbe312ZXiIiIqBx5r0cJhYeHw8XFBe7u7qplSqUSANCzZ09MmTIFTZo0wcyZM9G9e3eEhYUV2desWbOQlpamuiQmJuq8fiIiIiobWgUWa2tr6OvrIzk5WW15cnIybG1ti103KysLW7duxciRIwv0aWBggIYNG6otd3Z2LvYoIblcDgsLC7ULERERVUxaBRZDQ0O4ubkhMjJStUypVCIyMhKtWrUqdt3t27fj2bNn8Pf3L9BnixYtkJCQoLb82rVrcHBw0KY8IiIiqqAMtF1h6tSpCAwMRPPmzeHu7o7Q0FBkZWVh+PDhAICAgADUrFkTISEhauuFh4ejV69esLKyKtDn9OnTMXDgQLRt2xYdOnTA4cOH8fPPPyMqKqpke0VEREQVitaBZeDAgXj06BHmz5+PpKQkNGnSBIcPH1ZNxL137x709NQHbhISEvDbb7/h6NGjhfbZu3dvhIWFISQkBBMnTkSDBg2wc+dOtG7dugS7RERERBWN1udhkSqeh4WIiKj80cl5WIiIiIjKAgMLERERSR4DCxEREUkeAwsRERFJHgMLERERSR4DCxEREUkeAwsRERFJHgMLERERSR4DCxEREUkeAwsRERFJHgMLERERSR4DCxEREUkeAwsRERFJHgMLERERSR4DCxEREUkeAwsRERFJHgMLERERSR4DCxEREUkeAwsRERFJHgMLERERSR4DCxEREUkeAwsRERFJHgMLERERSZ5BWRdAREREQHZ2NuLj4zVun5OTgzt37kChUMDY2Fjj9ZycnGBiYlKSEssUAwsREZEExMfHw83NTefbiY6ORrNmzXS+ndLGwEJERKQD169fR0ZGhsbtc3JyEBERoXH727dvY968eVi0aBHq1Kmj1XZiYmI0bm9ubo769etr3F5XZEIIUdZFlIb09HRYWloiLS0NFhYWZV0OERF9wK5fvw5HR8eyLqPUXLt2TWehRdP3b46wEBERlbL8kZWIiAg4OzvrZBslncOijbi4OPj7+2s1UqQrDCxEREQ64uzsrNP5Il5eXjrrW2p4WDMRERFJHgMLERERSR6/EiIiIipluRmP0dRWD3fP7YPx02sarfPs2TM8ePBAx5UBNWrUgFwu16ht0u3baGqrB9nLXB1X9XYMLERERKUs+a/fEDPGDEhZDqRovl4TnVX0mkTNmzoD6DrGDPfEPzorR1MMLERERKWsTe+R2L0bUCgUMDIy0mgdKY6wAICpqSlqN+2kw4o0w/OwEBERUZnR9P2bk26JiIhI8hhYiIiISPIYWIiIiEjyGFiIiIhI8hhYiIiISPIYWIiIiEjyShRYVq5cqTq23MPDAxcuXCiybfv27SGTyQpcunXrVmj7sWPHQiaTITQ0tCSlERERUQWkdWDZtm0bpk6diuDgYMTExMDV1RU+Pj5ISSn8VH67du3Cw4cPVZfY2Fjo6+ujf//+Bdru3r0b586dQ40aNbTfEyIiIqqwtA4sy5YtQ1BQEIYPH46GDRsiLCwMJiYmWL9+faHtq1atCltbW9Xl2LFjMDExKRBY7t+/jwkTJmDz5s2oVKlSyfaGiIiIKiStAsvz588RHR0Nb2/v/3agpwdvb2+cPXtWoz7Cw8MxaNAgmJqaqpYplUoMHToU06dPR6NGjTTq59mzZ0hPT1e7EBERUcWkVWBJTU1FXl4ebGxs1Jbb2NggKSnpretfuHABsbGxGDVqlNrypUuXwsDAABMnTtS4lpCQEFhaWqou9vb2Gq9LRERE5ct7PUooPDwcLi4ucHd3Vy2Ljo7G999/j40bN0Imk2nc16xZs5CWlqa6JCZq8fOTREREVK5oFVisra2hr6+P5ORkteXJycmwtbUtdt2srCxs3boVI0eOVFt+6tQppKSkoHbt2jAwMICBgQHu3r2LadOmQaFQFNmfXC6HhYWF2oWIiIgqJgNtGhsaGsLNzQ2RkZHo1asXgFfzTyIjIzF+/Phi192+fTuePXsGf39/teVDhw5VmxMDAD4+Phg6dCiGDx+ucW35PzrNuSxERETlR/77dv77eFG0CiwAMHXqVAQGBqJ58+Zwd3dHaGgosrKyVOEiICAANWvWREhIiNp64eHh6NWrF6ysrNSWW1lZFVhWqVIl2NraokGDBhrXlZGRAQCcy0JERFQOZWRkwNLSssjrtQ4sAwcOxKNHjzB//nwkJSWhSZMmOHz4sGoi7r1796Cnp/5NU0JCAn777TccPXpU281prEaNGkhMTIS5ublWc2GkJD09Hfb29khMTORXXGWM94W08P6QDt4X0lFR7gshBDIyMt56DjaZeNsYDL036enpsLS0RFpaWrl+8FUEvC+khfeHdPC+kI4P7b7gbwkRERGR5DGwEBERkeQxsEiIXC5HcHAw5HJ5WZfyweN9IS28P6SD94V0fGj3BeewEBERkeRxhIWIiIgkj4GFiIiIJI+BhYiIiCSPgYWIiIgkj4GlFMhkMuzZs6esyyAiKpZCoUBoaGiptyV6HypEYBk2bBhkMhlkMhkqVaqEOnXq4H/+53+Qm5tb1qXp1Ov7/frlxo0bZVpT/g9jSm27f//9NwwNDdG4ceNCrz9x4gQ6duyIqlWrwsTEBPXr10dgYCCeP3+uarNu3Tq4urrCzMwMlStXRtOmTQv8btbjx48xefJkODg4wNDQEDVq1MCIESNw7969d97PiqCw+2rHjh0wMjLCd999p3pcf/XVV2pt9uzZo/azG1FRUZDJZGjUqBHy8vLU2lauXBkbN27U1S6Uqjdfv2xsbPDJJ59g/fr1UCqVpbqtixcvYvTo0aXetiSKev3KvygUCp1tW6oePXqEzz77DLVr14ZcLoetrS18fHxw4sQJWFtbF3hO5Fu0aBFsbGzw4sULbNy4ETKZDM7OzgXabd++vVzfthUisABAly5d8PDhQ9y6dQvLly/HmjVrEBwcXNZl6Vz+fr9+qVOnTon6ev2NuSLauHEjBgwYgPT0dJw/f17tuqtXr6JLly5o3rw5Tp48iT///BMrVqyAoaGh6s1w/fr1mDx5MiZOnIhLly7h9OnT+J//+R9kZmaq+nn8+DFatmyJX375BWFhYbhx4wa2bt2KGzduoEWLFrh169Z73efy4P/+7//g5+eH1atXY9q0aQAAIyMjLF26FE+ePHnr+rdu3cK//vUvXZepU/nP4zt37uDQoUPo0KEDJk2ahO7du+Ply5eltp1q1arBxMSk1NuWxPfff6/2ugUAGzZsUP198eJFtfYV/fUJAPr27Ys//vgDmzZtwrVr17Bv3z60b98eaWlp8Pf3x4YNGwqsI4TAxo0bERAQgEqVKgEATE1NkZKSgrNnz6q1DQ8PR+3atd/LvuiEqAACAwNFz5491Zb16dNHNG3aVPV3amqqGDRokKhRo4YwNjYWjRs3Flu2bFFbp127dmLChAli+vTpokqVKsLGxkYEBwertbl27Zpo06aNkMvlwtnZWRw9elQAELt371a1uXLliujQoYMwMjISVatWFUFBQSIjI6NAvYsXLxbVq1cXlpaWYsGCBeLFixfiiy++EFWqVBE1a9YU69ev13q/XxcVFSVatGghDA0Nha2trZgxY4Z48eKF2v6OGzdOTJo0SVhZWYn27dsLIYT4888/RZcuXYSpqamoXr268Pf3F48ePVKtt337dtG4cWPV/nXq1ElkZmaK4OBgAUDtcvz48WL3obS87bZQKpXio48+EocPHxYzZswQQUFBatcvX75cKBSKYrfRs2dPMWzYsGLbjB07VpiamoqHDx+qLc/OzhY1a9YUXbp0KX5HPgCv31dLly4VRkZGYteuXWrXd+/eXTg5OYnp06erlu/evVu8/pJ1/PhxAUBMnz5d2Nvbi9zcXNV1lpaWYsOGDTrfl9JQ1GM3MjJSABDr1q1TLXvy5IkYOXKksLa2Fubm5qJDhw7i0qVLauvt27dPNG/eXMjlcmFlZSV69eqlus7BwUEsX75cCPHqOREcHCzs7e2FoaGhsLOzExMmTCi0rRBC3L17V/To0UOYmpoKc3Nz0b9/f5GUlKS6Pjg4WLi6uop//etfwsHBQVhYWIiBAweK9PR0jW6HN19HHRwcxMKFC8XQoUOFubm5CAwMFEIIcerUKdG6dWthZGQkatWqJSZMmCAyMzNV6+Xm5opp06aJGjVqCBMTE+Hu7v7eXofexZMnTwQAERUVVej1V65cEQDEqVOn1JbnPw/i4uKEEEJs2LBBWFpaivHjx4tRo0ap2iUmJgq5XC5mzpwpHBwcdLYfulRhRlheFxsbizNnzsDQ0FC1LDc3F25ubjhw4ABiY2MxevRoDB06FBcuXFBbd9OmTTA1NcX58+fx9ddfY+HChTh27BgAQKlUok+fPjA0NMT58+cRFhaGGTNmqK2flZUFHx8fVKlSBRcvXsT27dvxyy+/YPz48Wrtfv31Vzx48AAnT57EsmXLEBwcjO7du6NKlSo4f/48xo4dizFjxuDvv/8u0W1w//59dO3aFS1atMDly5exevVqhIeH43//938L7K+hoSFOnz6NsLAwPH36FB07dkTTpk3x+++/4/Dhw0hOTsaAAQMAAA8fPsTgwYMxYsQIxMXFISoqCn369IEQAl988QUGDBigNurj6elZovpL2/Hjx5GdnQ1vb2/4+/tj69atyMrKUl1va2uLhw8f4uTJk0X2YWtri3PnzuHu3buFXq9UKrF161b4+fnB1tZW7TpjY2N8/vnnOHLkCB4/flw6O1XOzZgxA4sWLcL+/fvRu3dvtev09fWxZMkSrFix4q3PgcmTJ+Ply5dYsWKFLst97zp27AhXV1fs2rVLtax///5ISUnBoUOHEB0djWbNmqFTp06qx9SBAwfQu3dvdO3aFX/88QciIyPh7u5eaP87d+5UjUZfv34de/bsgYuLS6FtlUolevbsicePH+PEiRM4duwYbt26hYEDB6q1u3nzJvbs2YP9+/dj//79OHHiRJFfY2ji22+/haurK/744w/MmzcPN2/eRJcuXdC3b19cuXIF27Ztw2+//ab2+jp+/HicPXsWW7duxZUrV9C/f3906dIF169fL3Ed74OZmRnMzMywZ88ePHv2rMD1Li4uaNGiBdavX6+2fMOGDfD09ISTk5Pa8hEjRuA///kPsrOzAbwaYe7SpQtsbGx0txO6VtaJqTQEBgYKfX19YWpqKuRyuQAg9PT0xI4dO4pdr1u3bmLatGmqv9u1aydat26t1qZFixZixowZQgghjhw5IgwMDMT9+/dV1x86dEjtk8HatWtFlSpV1BL/gQMHhJ6enurTSGBgoHBwcBB5eXmqNg0aNBBt2rRR/f3y5Uthamoq/v3vf2u03/mXfv36CSGEmD17tmjQoIFQKpWq9itXrhRmZmaq7bZr105tFEoIIRYtWiQ6d+6stiwxMVEAEAkJCSI6OloAEHfu3CmypuJGOnTlbdsdMmSImDx5supvV1dXtU/gL1++FMOGDRMAhK2trejVq5dYsWKFSEtLU7V58OCBaNmypQAgHB0dRWBgoNi2bZvq9kxKShIA1D6Vvm7Xrl0CgDh//vw77Wt5FxgYKAwNDQUAERkZWej1+fdly5YtxYgRI4QQRY+wPHnyRISFhYmqVauKp0+fCiEqxgiLEEIMHDhQODs7CyFejSxYWFiojSQJIUTdunXFmjVrhBBCtGrVSvj5+RW5rddHTb777jvh6Ogonj9//ta2R48eFfr6+uLevXuq6//66y8BQFy4cEEI8WqExcTERG1EZfr06cLDw6PonX8NChlheX10SAghRo4cKUaPHq227NSpU0JPT0/k5OSIu3fvCn19fbXXaCGE6NSpk5g1a5ZGdZSlHTt2iCpVqggjIyPh6ekpZs2aJS5fvqy6PiwsTJiZmalG7NPT04WJiYn4v//7P1Wb/BEWIYRo0qSJ2LRpk1AqlaJu3bpi7969Yvny5RxhKWsdOnTApUuXcP78eQQGBmL48OHo27ev6vq8vDwsWrQILi4uqFq1KszMzHDkyJECEyE//vhjtb/t7OyQkpICAIiLi4O9vT1q1Kihur5Vq1Zq7ePi4uDq6gpTU1PVMi8vLyiVSiQkJKiWNWrUCHp6/735bWxs1D7d6Ovrw8rKSrXtt+13/uWHH35Q1dGqVSu1SYpeXl7IzMxU+8Tq5uam1t/ly5dx/PhxVdo3MzNTJfebN2/C1dUVnTp1gouLC/r3749169ZpNM+gLD19+hS7du2Cv7+/apm/vz/Cw8NVf+vr62PDhg34+++/8fXXX6NmzZpYsmQJGjVqpPp+3c7ODmfPnsWff/6JSZMm4eXLlwgMDESXLl3UJkcK/trFW3388cdQKBQIDg5WmwP0pqVLl2LTpk2Ii4srtr+RI0fCysoKS5cuLe1Sy5QQQvUcvnz5MjIzM2FlZaX2/Lx9+zZu3rwJALh06RI6deqkUd/9+/dHTk4OPvroIwQFBWH37t1FzpfJf+2zt7dXLWvYsCEqV66sdt8oFAqYm5ur/n799bMkmjdvrvb35cuXsXHjRrX99/HxgVKpxO3bt/Hnn38iLy8Pjo6Oam1OnDihuo2krG/fvnjw4AH27duHLl26ICoqCs2aNVNNIB88eDDy8vLwn//8BwCwbds26OnpFRjpyjdixAhs2LABJ06cQFZWFrp27fq+dkUnKkxgMTU1Rb169eDq6or169fj/Pnzam9I33zzDb7//nvMmDEDx48fx6VLl+Dj41NgIlf+pKV8Mpms1GfqF7Wdkmw7f7/zL3Z2dlrV8XqwAoDMzEz4+vqqhaBLly7h+vXraNu2LfT19XHs2DEcOnQIDRs2xIoVK9CgQQPcvn1bq+2+T1u2bEFubi48PDxgYGAAAwMDzJgxA7/99huuXbum1rZmzZoYOnQofvzxR/z111/Izc1FWFiYWpvGjRvj888/R0REBI4dO4Zjx47hxIkTqFatWoEX8NfFxcVBJpOhXr16OtvX8qJmzZqIiorC/fv30aVLF2RkZBTarm3btvDx8cGsWbOK7c/AwACLFy/G999/jwcPHuii5DIRFxenmkSfmZkJOzu7As/NhIQETJ8+HcCrrx41ZW9vj4SEBKxatUr1lWXbtm3x4sWLEtdb2q+fhb0+jRkzRm3/L1++jOvXr6Nu3brIzMyEvr4+oqOj1drExcXh+++/L3Ed75ORkRE++eQTzJs3D2fOnMGwYcNUB5BYWFigX79+qsm3GzZswIABA2BmZlZoX35+fjh37hy+/PJLDB06FAYGBu9tP3ShwgSW1+np6WH27NmYO3cucnJyAACnT59Gz5494e/vD1dXV3z00UcF3qzextnZGYmJiapP3ABw7ty5Am0uX76sNj/i9OnT0NPTQ4MGDd5hr7Tj7OyMs2fPqn3aP336NMzNzVGrVq0i12vWrBn++usvKBQKtSBUr1491YuHTCaDl5cXFixYgD/++AOGhobYvXs3AKgdVSMV4eHhmDZtWoEXuTZt2hT4Pvh1VapUgZ2dndp9+aaGDRsCeDV3SU9PDwMGDMCWLVuQlJSk1i4nJwerVq2Cj48PqlatWjo7Vs45ODjgxIkTSEpKKja0fPXVV/j5558LHPHwpv79+6NRo0ZYsGCBLsp973799Vf8+eefqpHiZs2aISkpCQYGBgWem9bW1gBejVxFRkZqvA1jY2P4+vrihx9+QFRUlGoE8U35r32JiYmqZVevXsXTp09Vz4H3oVmzZrh69WqB/a9Xrx4MDQ3RtGlT5OXlISUlpcD1b84rKy8aNmyo9ho0cuRI/Pbbb9i/fz/OnDmDkSNHFrlu1apV0aNHD5w4cQIjRox4H+XqVIUMLMCrFy99fX2sXLkSAFC/fn0cO3YMZ86cQVxcHMaMGYPk5GSt+vT29oajoyMCAwNx+fJlnDp1CnPmzFFr4+fnByMjIwQGBiI2NhbHjx/HhAkTMHTo0Pc62enzzz9HYmIiJkyYgPj4eOzduxfBwcGYOnWq2ldRbxo3bhweP36MwYMH4+LFi7h58yaOHDmC4cOHIy8vD+fPn8eSJUvw+++/4969e9i1axcePXqkOuZfoVDgypUrSEhIQGpq6jt9WtNWWlpagU+fp06dQkxMDEaNGoXGjRurXQYPHoxNmzbh5cuXWLNmDT777DMcPXoUN2/exF9//YUZM2bgr7/+gq+vLwDgs88+w6JFi3D69GncvXsX586dQ0BAAKpVq6b6anDJkiWwtbXFJ598gkOHDiExMREnT56Ej48PXrx4oXo80iv29vaIiopCSkoKfHx8kJ6eXqCNi4sL/Pz8VF93Fuerr77C+vXriw2ZUvTs2TMkJSXh/v37iImJwZIlS9CzZ090794dAQEBAF69/rRq1Qq9evXC0aNHcefOHZw5cwZz5szB77//DgAIDg7Gv//9bwQHByMuLg5//vlnkV+Tbdy4EeHh4YiNjcWtW7cQEREBY2NjODg4FGjr7e2tuh9iYmJw4cIFBAQEoF27dgW+ttGlGTNm4MyZMxg/frxq5Hfv3r2qSbeOjo7w8/NDQEAAdu3ahdu3b+PChQsICQnBgQMH3ludJfHPP/+gY8eOiIiIwJUrV3D79m1s374dX3/9NXr27Klq17ZtW9SrVw8BAQFwcnJ664ENGzduRGpqaoFJueVRhQ0sBgYGGD9+PL7++mtkZWVh7ty5aNasGXx8fNC+fXvY2tpqfYIzPT097N69Gzk5OXB3d8eoUaOwePFitTYmJiaqI0FatGiBfv36oVOnTvjxxx9Lce/ermbNmjh48CAuXLgAV1dXjB07FiNHjsTcuXOLXa9GjRo4ffo08vLy0LlzZ7i4uGDy5MmoXLky9PT0YGFhgZMnT6Jr165wdHTE3Llz8d133+HTTz8FAAQFBaFBgwZo3rw5qlWrhtOnT7+P3QXw6kRiTZs2VbusX78eDRs2LPTJ2rt3b6SkpODgwYNwd3dHZmYmxo4di0aNGqFdu3Y4d+4c9uzZg3bt2gF49aJ97tw59O/fH46Ojujbty+MjIwQGRkJKysrAICVlRXOnTuHDh06YMyYMahbty4GDBiAunXr4uLFi/joo4/e2+1RXtSqVQtRUVFITU0tMrQsXLhQo68WOnbsiI4dO5bquUveh8OHD8POzg4KhQJdunTB8ePH8cMPP2Dv3r3Q19cH8Gpk8+DBg2jbti2GDx8OR0dHDBo0CHfv3lV9GGrfvj22b9+Offv2oUmTJujYsWOBIyHzVa5cGevWrYOXlxc+/vhj/PLLL/j5559Vj+XXyWQy7N27F1WqVEHbtm3h7e2Njz76CNu2bdPdjVKIjz/+GCdOnMC1a9fQpk0bNG3aFPPnz1ebV7hhwwYEBARg2rRpaNCgAXr16oWLFy9K/vwjZmZm8PDwwPLly9G2bVs0btwY8+bNQ1BQkNr7h0wmw4gRI/DkyRONRk2MjY0LvU/LI5ngDEEiIiKSuAo7wkJEREQVBwMLERERSR4DCxEREUkeAwsRERFJHgMLERERSR4DCxEREUkeAwsRERFJHgMLERERSR4DCxEREUkeAwsRERFJHgMLERERSd7/Az4FBed4PEVdAAAAAElFTkSuQmCC",
            "text/plain": [
              "<Figure size 640x480 with 1 Axes>"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        }
      ],
      "source": [
        "matplotlib.pyplot.boxplot(results, labels=names)\n",
        "matplotlib.pyplot.title(\"Models' results' distribution of accuracy\")\n",
        "matplotlib.pyplot.show()"
      ]
    },
    {
      "attachments": {},
      "cell_type": "markdown",
      "metadata": {
        "id": "w6HrRE6U1YBS"
      },
      "source": [
        "### Generate the chosen model"
      ]
    },
    {
      "attachments": {},
      "cell_type": "markdown",
      "metadata": {
        "id": "oODOQLuOoFc2"
      },
      "source": [
        "Optimize the hyperparameters:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "PA6QOmgf1YQD",
        "outputId": "e9fe387f-7f2d-4434-bf4e-3028b5594ae6"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "The optimal hyperparameter 'C' is: 2.1060526315789474\n"
          ]
        }
      ],
      "source": [
        "model = lm.LogisticRegression(solver='liblinear', penalty='l1', max_iter=1000, random_state=config_dict['seed'])\n",
        "params = {\"C\": np.linspace(start=0.001, stop=10, num=20)}\n",
        "grid_search = GridSearchCV(model, params, scoring='accuracy')\n",
        "grid_search.fit(x_features_train, y_labels_train)\n",
        "print(\"The optimal hyperparameter 'C' is:\", grid_search.best_params_[\"C\"])\n"
      ]
    },
    {
      "attachments": {},
      "cell_type": "markdown",
      "metadata": {
        "id": "DjK2e9GaoO34"
      },
      "source": [
        "Fit the optimized model to the training set:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 74
        },
        "id": "KC8baVMOoPHk",
        "outputId": "c3af6863-cbaf-4c82-9ff3-deda1be6ca8a"
      },
      "outputs": [
        {
          "data": {
            "text/html": [
              "<style>#sk-container-id-1 {color: black;background-color: white;}#sk-container-id-1 pre{padding: 0;}#sk-container-id-1 div.sk-toggleable {background-color: white;}#sk-container-id-1 label.sk-toggleable__label {cursor: pointer;display: block;width: 100%;margin-bottom: 0;padding: 0.3em;box-sizing: border-box;text-align: center;}#sk-container-id-1 label.sk-toggleable__label-arrow:before {content: \"▸\";float: left;margin-right: 0.25em;color: #696969;}#sk-container-id-1 label.sk-toggleable__label-arrow:hover:before {color: black;}#sk-container-id-1 div.sk-estimator:hover label.sk-toggleable__label-arrow:before {color: black;}#sk-container-id-1 div.sk-toggleable__content {max-height: 0;max-width: 0;overflow: hidden;text-align: left;background-color: #f0f8ff;}#sk-container-id-1 div.sk-toggleable__content pre {margin: 0.2em;color: black;border-radius: 0.25em;background-color: #f0f8ff;}#sk-container-id-1 input.sk-toggleable__control:checked~div.sk-toggleable__content {max-height: 200px;max-width: 100%;overflow: auto;}#sk-container-id-1 input.sk-toggleable__control:checked~label.sk-toggleable__label-arrow:before {content: \"▾\";}#sk-container-id-1 div.sk-estimator input.sk-toggleable__control:checked~label.sk-toggleable__label {background-color: #d4ebff;}#sk-container-id-1 div.sk-label input.sk-toggleable__control:checked~label.sk-toggleable__label {background-color: #d4ebff;}#sk-container-id-1 input.sk-hidden--visually {border: 0;clip: rect(1px 1px 1px 1px);clip: rect(1px, 1px, 1px, 1px);height: 1px;margin: -1px;overflow: hidden;padding: 0;position: absolute;width: 1px;}#sk-container-id-1 div.sk-estimator {font-family: monospace;background-color: #f0f8ff;border: 1px dotted black;border-radius: 0.25em;box-sizing: border-box;margin-bottom: 0.5em;}#sk-container-id-1 div.sk-estimator:hover {background-color: #d4ebff;}#sk-container-id-1 div.sk-parallel-item::after {content: \"\";width: 100%;border-bottom: 1px solid gray;flex-grow: 1;}#sk-container-id-1 div.sk-label:hover label.sk-toggleable__label {background-color: #d4ebff;}#sk-container-id-1 div.sk-serial::before {content: \"\";position: absolute;border-left: 1px solid gray;box-sizing: border-box;top: 0;bottom: 0;left: 50%;z-index: 0;}#sk-container-id-1 div.sk-serial {display: flex;flex-direction: column;align-items: center;background-color: white;padding-right: 0.2em;padding-left: 0.2em;position: relative;}#sk-container-id-1 div.sk-item {position: relative;z-index: 1;}#sk-container-id-1 div.sk-parallel {display: flex;align-items: stretch;justify-content: center;background-color: white;position: relative;}#sk-container-id-1 div.sk-item::before, #sk-container-id-1 div.sk-parallel-item::before {content: \"\";position: absolute;border-left: 1px solid gray;box-sizing: border-box;top: 0;bottom: 0;left: 50%;z-index: -1;}#sk-container-id-1 div.sk-parallel-item {display: flex;flex-direction: column;z-index: 1;position: relative;background-color: white;}#sk-container-id-1 div.sk-parallel-item:first-child::after {align-self: flex-end;width: 50%;}#sk-container-id-1 div.sk-parallel-item:last-child::after {align-self: flex-start;width: 50%;}#sk-container-id-1 div.sk-parallel-item:only-child::after {width: 0;}#sk-container-id-1 div.sk-dashed-wrapped {border: 1px dashed gray;margin: 0 0.4em 0.5em 0.4em;box-sizing: border-box;padding-bottom: 0.4em;background-color: white;}#sk-container-id-1 div.sk-label label {font-family: monospace;font-weight: bold;display: inline-block;line-height: 1.2em;}#sk-container-id-1 div.sk-label-container {text-align: center;}#sk-container-id-1 div.sk-container {/* jupyter's `normalize.less` sets `[hidden] { display: none; }` but bootstrap.min.css set `[hidden] { display: none !important; }` so we also need the `!important` here to be able to override the default hidden behavior on the sphinx rendered scikit-learn.org. See: https://github.com/scikit-learn/scikit-learn/issues/21755 */display: inline-block !important;position: relative;}#sk-container-id-1 div.sk-text-repr-fallback {display: none;}</style><div id=\"sk-container-id-1\" class=\"sk-top-container\"><div class=\"sk-text-repr-fallback\"><pre>LogisticRegression(C=2.1060526315789474, max_iter=1000, random_state=0)</pre><b>In a Jupyter environment, please rerun this cell to show the HTML representation or trust the notebook. <br />On GitHub, the HTML representation is unable to render, please try loading this page with nbviewer.org.</b></div><div class=\"sk-container\" hidden><div class=\"sk-item\"><div class=\"sk-estimator sk-toggleable\"><input class=\"sk-toggleable__control sk-hidden--visually\" id=\"sk-estimator-id-1\" type=\"checkbox\" checked><label for=\"sk-estimator-id-1\" class=\"sk-toggleable__label sk-toggleable__label-arrow\">LogisticRegression</label><div class=\"sk-toggleable__content\"><pre>LogisticRegression(C=2.1060526315789474, max_iter=1000, random_state=0)</pre></div></div></div></div></div>"
            ],
            "text/plain": [
              "LogisticRegression(C=2.1060526315789474, max_iter=1000, random_state=0)"
            ]
          },
          "execution_count": 32,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "model = lm.LogisticRegression(C=grid_search.best_params_[\"C\"], max_iter=1000, random_state=config_dict['seed'])\n",
        "model.fit(x_features_train, y_labels_train)"
      ]
    },
    {
      "attachments": {},
      "cell_type": "markdown",
      "metadata": {
        "id": "ZinbMBQb8t5D"
      },
      "source": [
        "### Generate the ML train results: Use for Design Choices"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "J1A1nr-kSS0q",
        "outputId": "3ecd54ad-18f8-4ea5-ecf1-d994566d14b1"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "Results on the train set for a traditional ML model:\n",
            "-------------------------\n",
            "Baseline (dummy classifier) accuracy: 0.74\n",
            "Current model's accuracy: 0.88\n",
            "The accuracy lift is: 18 %\n"
          ]
        }
      ],
      "source": [
        "y_train_estimated = model.predict(x_features_train)\n",
        "accuracy_train = np.mean(y_train_estimated == y_labels_train)\n",
        "baseline_accuracy_train = np.mean(0 == y_labels_train)\n",
        "accuracy_lift_train = 100 * (accuracy_train/baseline_accuracy_train - 1)\n",
        "\n",
        "print(\"Results on the train set for a traditional ML model:\\n-------------------------\")\n",
        "print(\"Baseline (dummy classifier) accuracy:\", round(baseline_accuracy_train, 2))\n",
        "print(\"Current model's accuracy:\", round(accuracy_train, 2))\n",
        "print(\"The accuracy lift is:\", round(accuracy_lift_train), \"%\")"
      ]
    },
    {
      "attachments": {},
      "cell_type": "markdown",
      "metadata": {
        "id": "VRSrEo2j8y0P"
      },
      "source": [
        "### Generate the ML test results: Use for presenting performance"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "msAAgYoBSiK3",
        "outputId": "fab4e22b-0a32-4117-e740-21f9eac9a048"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "Results on the test set for a traditional ML model:\n",
            "-------------------------\n",
            "Baseline (dummy classifier) accuracy: 0.75\n",
            "Current model's accuracy: 0.85\n",
            "The accuracy lift is: 13 %\n",
            "\n",
            "Confusion Matrix:\n",
            "[[1506  126]\n",
            " [ 197  340]]\n",
            "\n",
            "Classification Report:\n",
            "              precision    recall  f1-score   support\n",
            "\n",
            "           0       0.88      0.92      0.90      1632\n",
            "           1       0.73      0.63      0.68       537\n",
            "\n",
            "    accuracy                           0.85      2169\n",
            "   macro avg       0.81      0.78      0.79      2169\n",
            "weighted avg       0.85      0.85      0.85      2169\n",
            "\n"
          ]
        }
      ],
      "source": [
        "y_test_estimated = model.predict(x_features_test)\n",
        "accuracy_test = np.mean(y_test_estimated == y_labels_test)\n",
        "baseline_accuracy_test = np.mean(0 == y_labels_test)\n",
        "accuracy_lift = 100 * (accuracy_test/baseline_accuracy_test - 1)\n",
        "\n",
        "print(\"Results on the test set for a traditional ML model:\\n-------------------------\")\n",
        "print(\"Baseline (dummy classifier) accuracy:\", round(baseline_accuracy_test, 2))\n",
        "print(\"Current model's accuracy:\", round(accuracy_test, 2))\n",
        "print(\"The accuracy lift is:\", round(accuracy_lift), \"%\")\n",
        "\n",
        "\n",
        "print(\"\\nConfusion Matrix:\")\n",
        "print(confusion_matrix(y_labels_test, y_test_estimated))\n",
        "print(\"\\nClassification Report:\")\n",
        "print(classification_report(y_labels_test, y_test_estimated))"
      ]
    },
    {
      "attachments": {},
      "cell_type": "markdown",
      "metadata": {
        "id": "s874Gqsa6XBi"
      },
      "source": [
        "****\n",
        "# Deep Learning  \n",
        "Applying BERT, a Language Model to Text Classification"
      ]
    },
    {
      "attachments": {},
      "cell_type": "markdown",
      "metadata": {
        "id": "6MYOOQ1gS2n2"
      },
      "source": [
        "## Formatting our data\n",
        "Adjusting the name of the label column:  \n",
        "The design of the Transformers package requires the dataset's lables column to be named exactly `label`.  \n",
        "In the above part of this notebook, where we did tranditional ML work, we had to pick a column name that **isn't** a natural word. The reason is that when we performed feature engineering, each word/Ngram was allocated its own column named after it. If the word \"label\" just happened to appear in the text, it could have a column called `label` defined for it in the dataframe, which would then **conflict with the labels' column name**.  \n",
        "We no longer have that risk, and we need to comply with Transformers' requirements:  "
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "VhT0emfy1rgP"
      },
      "outputs": [],
      "source": [
        "df = dataset_df_binary.copy()\n",
        "dataset_df_binary = dataset_df_binary.rename(columns={\"_label_\":\"label\"})"
      ]
    },
    {
      "attachments": {},
      "cell_type": "markdown",
      "metadata": {
        "id": "ZfY-cQ-nJGPs"
      },
      "source": [
        "Load the tokenizer and the pre-trained Language Model:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 388,
          "referenced_widgets": [
            "e1d893a202ae491fb11ecb066723d87b",
            "77bfef19bc9448a19a61c9aa816686df",
            "3e85747f79e3418db8dcdecb8ccc9e31",
            "050e326df53f410b8030e85c138f0ddf",
            "a6a6fc8a79ca4efca54e3aa635a0db16",
            "bc5b7870cae843d18666503afd6aac69",
            "cdfa0e4c66ab4d8995ec26bfc880ae04",
            "f9b6c024de4c4e5198d12d83a37ae755",
            "1b09e7ca7e46432e990084365804482d",
            "843f16eb13bc4d0ea2e7b694fbdaba89",
            "ba78731b08e744e7a3fb8ef2f14635d6",
            "0aff96d59e384791a9772374e1421869",
            "d414285c2c164ee295122978b6fb160c",
            "cc3a62d71b984b40a89b0bdced53c44c",
            "28ca2f694a354d809a915386bcec72b8",
            "54a03d9092b040adbfbb04a95032cd74",
            "1f7bff426453491b99c5627165929011",
            "b6833c3bbf5a4e9da2acafdc1c9829d8",
            "6fac132c8e774a4d9c59c0b66e380f5d",
            "a3cc1264338346a2b09c00d193e6fff8",
            "9c4dea561e9a45f18b77488e3a2c8c38",
            "caa50f95551e4abd8bb8f910d3dcc542",
            "eb5cfe8520104bf79843ac0608b2fe0a",
            "2ab539e2c1894c9d93913a03700701e3",
            "040e9974eb8b4ee287b6eb8ca4536b78",
            "799c054bca4f4f35a65454e255096a4d",
            "49f5db556509467cb64fa6fb0fb57cb3",
            "20116e05c09348c8894ad804a93f8123",
            "55561deb9e564557bfabfead3dcf43cb",
            "b288c1bf38d3494b975985f204b646e3",
            "b5e951d4a051424d9c28ec4ac9a7ac57",
            "474054f4c4b64600ab8216e104d234bf",
            "3ddce4f62bc243bbae17fbc754c74932",
            "2af5480eafce4ce5a181b119c57ae231",
            "2cd51affbd3e4c9a9d42deae61ef8725",
            "3ff75e0b9dd34817bcffa6c6d4d0cf66",
            "c65103ea708c4494b075f792ec7d2578",
            "103a2c2a91ee41e8a9f2178decfa773c",
            "3f81f3084468484e8f901d09c2ea5b07",
            "994f3c155dc44c27a978785a7a15bbfe",
            "ebc84f172be540e498eab0ac92e98879",
            "664e5b569b914d7fbde69b93d761625c",
            "c1961a2b423647e7b9e5faa3edf75db0",
            "16350733c6ca4eff8611df6d6bcc00dd",
            "810b971582fe4616b42cb0706871adef",
            "c626d329a7894f85ab044ba5b746171d",
            "78df8e9bda114ecb9d78475ef64efd16",
            "c42836171b794b7683f9613593128b16",
            "dcd4877ccf2f4115b666f3c9e82a8c54",
            "394e3053b6184980ad8b1cf9ac883870",
            "f1eba7e9fc6d4fc892984c947cc2ec00",
            "098c142a3016470b981240d818e681a5",
            "c9cdd34643c24241b567a053bb2fd562",
            "99b503702cfa4282b46d92d19cfc841f",
            "7099cb7aa89f482e843d17ca5548f597"
          ]
        },
        "id": "WCCvrZYiJGPs",
        "outputId": "08114752-0bbc-426a-cc80-13d8e8de416e"
      },
      "outputs": [
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "/usr/local/lib/python3.10/dist-packages/huggingface_hub/utils/_token.py:88: UserWarning: \n",
            "The secret `HF_TOKEN` does not exist in your Colab secrets.\n",
            "To authenticate with the Hugging Face Hub, create a token in your settings tab (https://huggingface.co/settings/tokens), set it as secret in your Google Colab and restart your session.\n",
            "You will be able to reuse this secret in all of your notebooks.\n",
            "Please note that authentication is recommended but still optional to access public models or datasets.\n",
            "  warnings.warn(\n"
          ]
        },
        {
          "data": {
            "application/vnd.jupyter.widget-view+json": {
              "model_id": "e1d893a202ae491fb11ecb066723d87b",
              "version_major": 2,
              "version_minor": 0
            },
            "text/plain": [
              "tokenizer_config.json:   0%|          | 0.00/48.0 [00:00<?, ?B/s]"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        },
        {
          "data": {
            "application/vnd.jupyter.widget-view+json": {
              "model_id": "0aff96d59e384791a9772374e1421869",
              "version_major": 2,
              "version_minor": 0
            },
            "text/plain": [
              "config.json:   0%|          | 0.00/570 [00:00<?, ?B/s]"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        },
        {
          "data": {
            "application/vnd.jupyter.widget-view+json": {
              "model_id": "eb5cfe8520104bf79843ac0608b2fe0a",
              "version_major": 2,
              "version_minor": 0
            },
            "text/plain": [
              "vocab.txt:   0%|          | 0.00/232k [00:00<?, ?B/s]"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        },
        {
          "data": {
            "application/vnd.jupyter.widget-view+json": {
              "model_id": "2af5480eafce4ce5a181b119c57ae231",
              "version_major": 2,
              "version_minor": 0
            },
            "text/plain": [
              "tokenizer.json:   0%|          | 0.00/466k [00:00<?, ?B/s]"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        },
        {
          "data": {
            "application/vnd.jupyter.widget-view+json": {
              "model_id": "810b971582fe4616b42cb0706871adef",
              "version_major": 2,
              "version_minor": 0
            },
            "text/plain": [
              "model.safetensors:   0%|          | 0.00/440M [00:00<?, ?B/s]"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForSequenceClassification: ['cls.predictions.transform.LayerNorm.weight', 'cls.seq_relationship.bias', 'cls.predictions.transform.LayerNorm.bias', 'cls.predictions.transform.dense.bias', 'cls.predictions.transform.dense.weight', 'cls.predictions.bias', 'cls.seq_relationship.weight']\n",
            "- This IS expected if you are initializing BertForSequenceClassification from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n",
            "- This IS NOT expected if you are initializing BertForSequenceClassification from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n",
            "Some weights of BertForSequenceClassification were not initialized from the model checkpoint at bert-base-uncased and are newly initialized: ['classifier.weight', 'classifier.bias']\n",
            "You should probably TRAIN this model on a down-stream task to be able to use it for predictions and inference.\n"
          ]
        }
      ],
      "source": [
        "labels = list(dataset_df_binary[\"label\"].unique())\n",
        "\n",
        "from transformers import AutoTokenizer, AutoModelForSequenceClassification\n",
        "tokenizer = AutoTokenizer.from_pretrained(\"bert-base-uncased\")\n",
        "language_model = AutoModelForSequenceClassification.from_pretrained(\"bert-base-uncased\", num_labels=len(labels))"
      ]
    },
    {
      "attachments": {},
      "cell_type": "markdown",
      "metadata": {
        "id": "XYECTcgS6XlH"
      },
      "source": [
        ">>\n",
        "The previous cell would output a warnin starting with:  \n",
        "`Some weights of the model checkpoint at bert-base-uncased were not used when initializing...`\n",
        "\n",
        ">>\n",
        "It is expected, as the model you imported had its pre-trained classification head (i.e. last neural layer) removed and a new \"fresh\" layer is initialized.  \n",
        "That's what we want, as we seek to train that classification head to suit our dataset. Based on our choice, we may choose to also fine-tune other layers.    "
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "s5TAnpQy8ww6",
        "outputId": "5e333041-01d1-47d0-b6aa-c5088d75bd14"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "The size of the model's token dictionary: 30522\n"
          ]
        }
      ],
      "source": [
        "print(f\"The size of the model's token dictionary: {language_model.config.vocab_size}\")"
      ]
    },
    {
      "attachments": {},
      "cell_type": "markdown",
      "metadata": {
        "id": "_VSj4AUH3el6"
      },
      "source": [
        "Split the dataset to three sub-sets:  \n",
        "1. A held-out test set  \n",
        "2. A train set that is split to two:  \n",
        "  2.1 A subset used for training the neural network's parameters  \n",
        "  2.2 A subset used to evaluate the progress of the training  "
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "EJQx35vq3Wkt"
      },
      "outputs": [],
      "source": [
        "# Create a training set and a test set\n",
        "test_df = dataset_df_binary.sample(frac=config_dict[\"test_size\"],random_state=config_dict['seed'])\n",
        "train_df = dataset_df_binary.drop(test_df.index)\n",
        "\n",
        "# Splitting the train set to \"just train\" and \"training evaluation\" set:\n",
        "train_eval_df = train_df.sample(frac=config_dict[\"test_size\"],random_state=config_dict['seed'])\n",
        "train_train_df = train_df.drop(train_eval_df.index)\n",
        "\n",
        "# Conver the dataframes to a Dataset format per the Transformers package's requirement:\n",
        "dataset_train_train = Dataset.from_pandas(train_train_df)\n",
        "dataset_train_eval = Dataset.from_pandas(train_eval_df)\n",
        "dataset_test = Dataset.from_pandas(test_df)"
      ]
    },
    {
      "attachments": {},
      "cell_type": "markdown",
      "metadata": {
        "id": "rI25Mza34Qut"
      },
      "source": [
        "In order for the LM to process the text, it must be tokenized:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 113,
          "referenced_widgets": [
            "8f160746113648e4af8bc53395aedf0f",
            "ef5d2d6ad6804c4499ca15001c945eb6",
            "17bd1279c68a42e8bb9b8c391c8cfb15",
            "45b4f32b3fc54e1dac70fa3a822e4452",
            "535eb7a9b346475ab686fa25ae93f2f7",
            "313bd6d39eef44b1a998bcea69cec16b",
            "26c7c8a8544f4f0a9e0412b3bea5f5e5",
            "86f3780520254be8a69ba0c9ecc93f1c",
            "74cb7283bb5042dcb8ac3eca570cb851",
            "7a03dac8569b41dd949d6ba8bac22326",
            "d1ae726bcad24b579a39735109bdb4d5",
            "d579c0801d51437f808ecc022aa08597",
            "7f423470b4904dd1a01391f896dfd5d6",
            "5120a915db65442893ffceb18b19c751",
            "e3285409f724447ea8d24c5303f759ae",
            "3bf1190a3fd8497d9344dd5240b4e57d",
            "e98460d193cc4431abff6a1f68776b5e",
            "dc6a773fa8a548bfbfe66f3036f165c0",
            "5702cfd040c441c09f9f07696555b910",
            "48f7aaf647894e2a94ffbbeea565d246",
            "27737dab270a42518eefb5ec03413425",
            "9c794b66c71e419494985a23df365b3c",
            "b5843ae0eb13403780d7d5856e99cde9",
            "03eca2f571d74759b1453f03e28ee360",
            "f08897fd22c6466fa3217ee5bd5e3a57",
            "ea36dc4bc5014f9c852c34a2275337f0",
            "4daebb75a50b4f52a2e15f333ca90609",
            "87b77cd65ec74d769cfaf2f056946a58",
            "40c1a35ae41048a5a36aaaa5517d232e",
            "8e154cbae8e540fba9ad5c3ff7f86514",
            "57ad49ce27e3403398e3256d9d9b8ca8",
            "360ae5e4d031449681de6d5165c0cfe6",
            "dd0d83b72beb4ec1a383e52d8b9a39ec"
          ]
        },
        "id": "4WfTVCYm4Q6V",
        "outputId": "1954a0ae-1138-4d93-9118-b86ec8b26f60"
      },
      "outputs": [
        {
          "data": {
            "application/vnd.jupyter.widget-view+json": {
              "model_id": "8f160746113648e4af8bc53395aedf0f",
              "version_major": 2,
              "version_minor": 0
            },
            "text/plain": [
              "Map:   0%|          | 0/4880 [00:00<?, ? examples/s]"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        },
        {
          "data": {
            "application/vnd.jupyter.widget-view+json": {
              "model_id": "d579c0801d51437f808ecc022aa08597",
              "version_major": 2,
              "version_minor": 0
            },
            "text/plain": [
              "Map:   0%|          | 0/1627 [00:00<?, ? examples/s]"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        },
        {
          "data": {
            "application/vnd.jupyter.widget-view+json": {
              "model_id": "b5843ae0eb13403780d7d5856e99cde9",
              "version_major": 2,
              "version_minor": 0
            },
            "text/plain": [
              "Map:   0%|          | 0/2169 [00:00<?, ? examples/s]"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        }
      ],
      "source": [
        "def tokenize_function(examples):\n",
        "    return tokenizer(examples[\"text\"], padding=\"max_length\", truncation=True)\n",
        "\n",
        "train_train_tokenized = dataset_train_train.map(tokenize_function, batched=True)\n",
        "train_eval_tokenized = dataset_train_eval.map(tokenize_function, batched=True)\n",
        "test_tokenized = dataset_test.map(tokenize_function, batched=True)"
      ]
    },
    {
      "attachments": {},
      "cell_type": "markdown",
      "metadata": {
        "id": "r05gJRSqJGPu"
      },
      "source": [
        "<a id='trainer'></a>"
      ]
    },
    {
      "attachments": {},
      "cell_type": "markdown",
      "metadata": {
        "id": "2ijS9JahJGPv"
      },
      "source": [
        "## Training our LM"
      ]
    },
    {
      "attachments": {},
      "cell_type": "markdown",
      "metadata": {
        "id": "7gFV8yDTJGPv"
      },
      "source": [
        "We fine tune our pre-trained Language Model via `transformers`'s [Trainer](https://huggingface.co/docs/transformers/main/en/main_classes/trainer#transformers.Trainer).  "
      ]
    },
    {
      "attachments": {},
      "cell_type": "markdown",
      "metadata": {
        "id": "xL_AepCYGi19"
      },
      "source": [
        "### Choosing which neural network layers to fine-tune"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "umQ7F-mvGjAl",
        "outputId": "a230a48a-e40f-43b2-c4be-32fa70339ef2"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "Fine-tuning the entire neural network!\n"
          ]
        }
      ],
      "source": [
        "if layers_to_fine_tune == \"head\":\n",
        "  print(\"Fine-tuning only the classification head!\")\n",
        "  language_model.train()\n",
        "  for name, param in language_model.named_parameters():\n",
        "    # Freeze parameters of all layers except classifier head:\n",
        "    if 'classifier' not in name:\n",
        "        param.requires_grad = False\n",
        "else:\n",
        "  print(\"Fine-tuning the entire neural network!\")"
      ]
    },
    {
      "attachments": {},
      "cell_type": "markdown",
      "metadata": {
        "id": "ZWADrXqRJGPw"
      },
      "source": [
        "### Training hyperparameters"
      ]
    },
    {
      "attachments": {},
      "cell_type": "markdown",
      "metadata": {
        "id": "K7jPvmmVJGPw"
      },
      "source": [
        "These are the settings for training our model:"
      ]
    },
    {
      "attachments": {},
      "cell_type": "markdown",
      "metadata": {
        "id": "xI5pi9fnJGPw"
      },
      "source": [
        "### Evaluation metric"
      ]
    },
    {
      "attachments": {},
      "cell_type": "markdown",
      "metadata": {
        "id": "GFGxObdeJGPx"
      },
      "source": [
        "Defining the evaluation metric for the Language Model fine-tuning:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 49,
          "referenced_widgets": [
            "4e59624846fe48edbf91475a2b7fdebf",
            "97ab2101ef8149cebbe40aa0b7169d3a",
            "c9482ecfd4004fa9ac8602f782f249d6",
            "fa48b2a69f934782a8972b00a4dba8fc",
            "160b3e8a0f9144dbb30aa062eb296686",
            "cd9098d89fab4e519290a90133c6e1cd",
            "d465ae1a0df44d85bc241da6a5810e0a",
            "694acd04ca7e4eb6b9f1ee5308ce1890",
            "98274506b9434f15a3046703f6316bc4",
            "fd8f0c3b6f9944dfa21fb1613a839538",
            "6d2948debccd4ca9841a0a1175120f65"
          ]
        },
        "id": "3tFbTVfpJGPx",
        "outputId": "f659b016-ee96-4a7b-a539-7563ed510ab5"
      },
      "outputs": [
        {
          "data": {
            "application/vnd.jupyter.widget-view+json": {
              "model_id": "4e59624846fe48edbf91475a2b7fdebf",
              "version_major": 2,
              "version_minor": 0
            },
            "text/plain": [
              "Downloading builder script:   0%|          | 0.00/4.20k [00:00<?, ?B/s]"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        }
      ],
      "source": [
        "metric = evaluate.load(\"accuracy\")"
      ]
    },
    {
      "attachments": {},
      "cell_type": "markdown",
      "metadata": {
        "id": "Q3A9_OZtJGPx"
      },
      "source": [
        "Setting the metric evaluation function for the trainer to utilize:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "xUPhu4n_JGPx"
      },
      "outputs": [],
      "source": [
        "def compute_metrics(eval_pred):\n",
        "    logits, labels = eval_pred\n",
        "    # As the model returns a pair of logit values for each observation,\n",
        "    # where each of the two logit value reflects the likelihood of each\n",
        "    # class, we want to conver it to a classification:\n",
        "    predictions = np.argmax(logits, axis=-1)\n",
        "    return metric.compute(predictions=predictions, references=labels)"
      ]
    },
    {
      "attachments": {},
      "cell_type": "markdown",
      "metadata": {
        "id": "_bowLe51JGPy"
      },
      "source": [
        "### Trainer object"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "SGEfjqqWJGPy"
      },
      "outputs": [],
      "source": [
        "trainer = Trainer(\n",
        "    model=language_model,\n",
        "    args=lm_training_args,\n",
        "    train_dataset=train_train_tokenized,\n",
        "    eval_dataset=train_eval_tokenized,\n",
        "    compute_metrics=compute_metrics,\n",
        ")"
      ]
    },
    {
      "attachments": {},
      "cell_type": "markdown",
      "metadata": {
        "id": "49tZDZCv8-yT"
      },
      "source": [
        "### Fine tuning"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 541
        },
        "id": "rFChSNaqJGPz",
        "outputId": "9bd8fe46-3236-420f-94c6-f0673622a131"
      },
      "outputs": [
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "/usr/local/lib/python3.10/dist-packages/transformers/optimization.py:391: FutureWarning: This implementation of AdamW is deprecated and will be removed in a future version. Use the PyTorch implementation torch.optim.AdamW instead, or set `no_deprecation_warning=True` to disable this warning\n",
            "  warnings.warn(\n"
          ]
        },
        {
          "data": {
            "text/html": [
              "\n",
              "    <div>\n",
              "      \n",
              "      <progress value='1220' max='1220' style='width:300px; height:20px; vertical-align: middle;'></progress>\n",
              "      [1220/1220 24:29, Epoch 2/2]\n",
              "    </div>\n",
              "    <table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              " <tr style=\"text-align: left;\">\n",
              "      <th>Step</th>\n",
              "      <th>Training Loss</th>\n",
              "      <th>Validation Loss</th>\n",
              "      <th>Accuracy</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <td>100</td>\n",
              "      <td>0.432400</td>\n",
              "      <td>0.290901</td>\n",
              "      <td>0.880148</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>200</td>\n",
              "      <td>0.264100</td>\n",
              "      <td>0.229594</td>\n",
              "      <td>0.921942</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>300</td>\n",
              "      <td>0.249000</td>\n",
              "      <td>0.228554</td>\n",
              "      <td>0.927474</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>400</td>\n",
              "      <td>0.246900</td>\n",
              "      <td>0.309618</td>\n",
              "      <td>0.911494</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>500</td>\n",
              "      <td>0.201800</td>\n",
              "      <td>0.223265</td>\n",
              "      <td>0.938537</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>600</td>\n",
              "      <td>0.186800</td>\n",
              "      <td>0.253884</td>\n",
              "      <td>0.940381</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>700</td>\n",
              "      <td>0.110000</td>\n",
              "      <td>0.272044</td>\n",
              "      <td>0.936693</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>800</td>\n",
              "      <td>0.091600</td>\n",
              "      <td>0.279178</td>\n",
              "      <td>0.939152</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>900</td>\n",
              "      <td>0.114100</td>\n",
              "      <td>0.245325</td>\n",
              "      <td>0.942840</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>1000</td>\n",
              "      <td>0.097500</td>\n",
              "      <td>0.251047</td>\n",
              "      <td>0.944683</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>1100</td>\n",
              "      <td>0.120200</td>\n",
              "      <td>0.243593</td>\n",
              "      <td>0.944069</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>1200</td>\n",
              "      <td>0.137800</td>\n",
              "      <td>0.234556</td>\n",
              "      <td>0.945298</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table><p>"
            ],
            "text/plain": [
              "<IPython.core.display.HTML object>"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        },
        {
          "data": {
            "text/plain": [
              "TrainOutput(global_step=1220, training_loss=0.18632200194186851, metrics={'train_runtime': 1472.0629, 'train_samples_per_second': 6.63, 'train_steps_per_second': 0.829, 'total_flos': 2567963900313600.0, 'train_loss': 0.18632200194186851, 'epoch': 2.0})"
            ]
          },
          "execution_count": 44,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "trainer.train()"
      ]
    },
    {
      "attachments": {},
      "cell_type": "markdown",
      "metadata": {
        "id": "vV1wOxvzAdKh"
      },
      "source": [
        "Converting the training log to a dataframe for plotting:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "QgNw76aIAvh-"
      },
      "outputs": [],
      "source": [
        "training_logs_df = pd.DataFrame(trainer.state.log_history).groupby(\"step\", as_index=False).first()"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 472
        },
        "id": "3QCwRW8bAdY7",
        "outputId": "71b75de3-e09e-4821-89b5-0652a9ee9e36"
      },
      "outputs": [
        {
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAiwAAAHHCAYAAACcHAM1AAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjcuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/bCgiHAAAACXBIWXMAAA9hAAAPYQGoP6dpAACPPUlEQVR4nOzdd3RU1RbA4d+k90ZIg5BG7xBI6CAEAiJSBXl0BSsqIvrsgKKIIqIUUZ6KgErHLiCRIhCK9C4lBAIk1HRS57w/rhkZkkASkswk7G+tWZm5c+fcfafunKpTSimEEEIIIcyYhakDEEIIIYS4E0lYhBBCCGH2JGERQgghhNmThEUIIYQQZk8SFiGEEEKYPUlYhBBCCGH2JGERQgghhNmThEUIIYQQZk8SFiGEEEKYPUlYytCkSZPQ6XRcuXLF1KEUy5kzZ9DpdCxYsMDUoRgJDAzkgQceMHUYpS4hIYEBAwZQpUoVdDodM2fONHVIogQq6uf9VoGBgYwcObJEj+3UqROdOnUq1XiEyCMJSzEdPnyYoUOHUq1aNWxtbfHz82PIkCEcPnzY1KFVCkeOHGHSpEmcOXPG1KGUm+eff561a9fyyiuvsGjRIrp3727qkMRtvPvuu3z//fcmO/62bduYNGkSiYmJJotBCFOwMnUAFcmqVasYPHgwHh4ePProowQFBXHmzBm++OILVqxYwZIlS+jbt6+pw7xrAQEB3LhxA2tr63I/9pEjR5g8eTKdOnUiMDCw3I9vCn/88Qe9e/dmwoQJpg5FFMG7777LgAED6NOnj0mOv23bNiZPnszIkSNxc3Mr9fKPHz+OhUXJ/pddt25dKUcjxL8kYSmiU6dOMWzYMIKDg9m8eTNVq1Y13Pfcc8/Rvn17hg0bxoEDBwgODjZhpPnp9XqysrKws7Mr0v46na7I+4qSycnJQa/XY2Njw6VLl0r1hycjIwMbG5sS/+iIyqO4n30AW1vbEh/PxsamxI81pbS0NBwdHSvdsSob+UYrog8++ID09HQ+//xzo2QFwNPTk88++4y0tDTef//9fI+9cuUKAwcOxMXFhSpVqvDcc8+RkZFhtM/vv/9Ou3btcHNzw8nJiTp16vDqq68a7ZOZmcnEiROpWbMmtra2+Pv789JLL5GZmWm0n06nY+zYsXzzzTc0aNAAW1tbfvrpJzw8PBg1alS++JKTk7GzszP8h19QH5aRI0fi5OTE+fPn6dOnD05OTlStWpUJEyaQm5trVN7Vq1cZNmwYLi4uuLm5MWLECPbv33/HfjELFizgoYceAuC+++5Dp9Oh0+nYuHGj0X5btmwhLCwMOzs7goODWbhwYb6yEhMTGTduHP7+/tja2lKzZk2mTZuGXq8v9Ph58vrKrFu3jqZNm2JnZ0f9+vVZtWpViY6T93xOnz6dmTNnEhISgq2tLXPnzkWn06GUYs6cOYbzzXP69GkeeughPDw8cHBwoFWrVvzyyy9Gx9+4cSM6nY4lS5bw+uuvU61aNRwcHEhOTja8ZmfPnuWBBx7AycmJatWqMWfOHAAOHjxI586dcXR0JCAggG+//dao7GvXrjFhwgQaNWqEk5MTLi4u9OjRg/379xcYw7Jly3jnnXeoXr06dnZ2dOnShZMnT+Z7znbs2MH999+Pu7s7jo6ONG7cmI8//thon2PHjjFgwAA8PDyws7OjRYsW/Pjjj3d87QCmT59OmzZtqFKlCvb29oSGhrJixYp8+xXlM3crnU5HWloaX3/9teH1urW/R2JioqH2w9XVlVGjRpGenp6vrMWLFxMaGoq9vT0eHh48/PDDnDt37rbHnzRpEi+++CIAQUFBhhjymlAL+uyvWbOmWM/LrX1YFixYgE6nY+vWrYwfP56qVavi6OhI3759uXz5stFjb+3DUtz3xpw5cwgODsbe3p6wsDD+/PPPIveLufnc69Spg52dHaGhoWzevDnfc6jT6Thy5Aj/+c9/cHd3p127doD2j8Tbb79t+IwGBgby6quv5vuO1ev1TJo0CT8/PxwcHLjvvvs4cuRIoc/dpk2beOqpp/Dy8qJ69eqG+3/77Tfat2+Po6Mjzs7O9OzZM1/3gvj4eEaNGkX16tWxtbXF19eX3r17GzWb//XXX0RGRuLp6Ym9vT1BQUE88sgj+WKeOXMmDRo0wM7ODm9vbx5//HGuX79utF9RyjIZJYrEz89PBQYG3nafwMBAVb16dcPtiRMnKkA1atRI9erVS82ePVsNHTpUAWrYsGGG/Q4dOqRsbGxUixYt1Mcff6zmzZunJkyYoDp06GDYJzc3V3Xr1k05ODiocePGqc8++0yNHTtWWVlZqd69exvFAah69eqpqlWrqsmTJ6s5c+aovXv3qkceeUS5ubmpzMxMo/2//vprBahdu3YppZSKiYlRgPrqq68M+4wYMULZ2dmpBg0aqEceeUR9+umnqn///gpQc+fONYqzdevWytLSUo0dO1bNnj1bde3aVTVp0iRfmbc6deqUevbZZxWgXn31VbVo0SK1aNEiFR8fr5RSKiAgQNWpU0d5e3urV199Vc2ePVs1b95c6XQ6dejQIUM5aWlpqnHjxqpKlSrq1VdfVfPmzVPDhw9XOp1OPffcc7d9DfOOU7t2beXm5qZefvllNWPGDNWoUSNlYWGh1q1bV+zj5D2f9evXV8HBweq9995TH330kdq0aZNatGiRAlTXrl0N56uUUvHx8crb21s5Ozur1157Tc2YMUM1adJEWVhYqFWrVhnK3rBhg6Hspk2bqhkzZqipU6eqtLQ0w2tWv3599cQTT6g5c+aoNm3aGF4HPz8/9eKLL6pZs2apBg0aKEtLS3X69GlD2bt27VIhISHq5ZdfVp999pl66623VLVq1ZSrq6s6f/58vhiaNWumQkND1UcffaQmTZqkHBwcVFhYmNFzu27dOmVjY6MCAgLUxIkT1aeffqqeffZZFRERYdjn0KFDytXVVdWvX19NmzZNzZ49W3Xo0EHpdDqjcy9M9erV1VNPPaVmz56tZsyYocLCwhSgfv75Z6Nj3OkzV5BFixYpW1tb1b59e8PrtW3bNqXUv5/3Zs2aqX79+qm5c+eq0aNHK0C99NJLRuVMmTJF6XQ6NWjQIDV37lw1efJk5enpqQIDA9X169cLPf7+/fvV4MGDFaA++ugjQwypqalKqcI/+0V9XpTS3v8jRoww3P7qq68M59W5c2c1a9Ys9cILLyhLS0s1cOBAo8d27NhRdezY0XC7OO+NuXPnKkC1b99effLJJ2r8+PHKw8NDhYSEGJVZGEA1bNhQeXp6qrfeektNmzZNBQQEKHt7e3Xw4EHDfnmvU/369VXv3r3V3Llz1Zw5c5RS2vccoAYMGKDmzJmjhg8frgDVp08fo2O99NJLCjB8r48ZM0ZVr15deXp6Fvjc1a9fX3Xs2FHNmjVLvffee0oppRYuXKh0Op3q3r27mjVrlpo2bZoKDAxUbm5uKiYmxlBGmzZtlKurq3r99dfV//73P/Xuu++q++67T23atEkppVRCQoJyd3dXtWvXVh988IGaP3++eu2111S9evWMYh49erSysrJSY8aMUfPmzVP//e9/laOjo2rZsqXKysoqVlmmIglLESQmJiogX2JwqwcffFABKjk5WSn17wfjwQcfNNrvqaeeUoDav3+/Ukqpjz76SAHq8uXLhZa9aNEiZWFhof7880+j7fPmzVOA2rp1q2EboCwsLNThw4eN9l27dq0C1E8//WS0/f7771fBwcGG24UlLIB66623jB6b90WUZ+XKlQpQM2fONGzLzc1VnTt3vmPCopRSy5cvV4DasGFDvvsCAgIUoDZv3mzYdunSJWVra6teeOEFw7a3335bOTo6qr///tvo8S+//LKytLRUZ8+evW0MecdZuXKlYVtSUpLy9fVVzZo1K/Zx8p5PFxcXdenSpXzHA9TTTz9ttG3cuHEKMHq9U1JSVFBQkAoMDFS5ublKqX9/EIKDg1V6erpRGXmv2bvvvmvYdv36dWVvb690Op1asmSJYfuxY8cUoCZOnGjYlpGRYThOnpiYGGVra2v0PsiLoV69ekbJ8Mcff6wAw49FTk6OCgoKUgEBAfl+lPV6veF6ly5dVKNGjVRGRobR/W3atFG1atXK9/zd6tbnISsrSzVs2FB17tzZsK0on7nCODo6Gv0o5cn7vD/yyCNG2/v27auqVKliuH3mzBllaWmp3nnnHaP9Dh48qKysrPJtv9UHH3ygAKMftTyFffaVKtrzolThCUtERITR6/T8888rS0tLlZiYaNhWWMJyp/dGZmamqlKlimrZsqXKzs427LdgwQIFFDlhAdRff/1l2BYbG6vs7OxU3759DdvyXqfBgwcbPX7fvn0KUKNHjzbaPmHCBAWoP/74Qyml/TNhZWWVL4mZNGmSAgp87tq1a6dycnIM21NSUpSbm5saM2aMURnx8fHK1dXVsP369esKUB988EGh57169WqjfzgL8ueffypAffPNN0bb16xZY7S9KGWZkjQJFUFKSgoAzs7Ot90v7/7k5GSj7U8//bTR7WeeeQaAX3/9FcDQf+GHH34otMli+fLl1KtXj7p163LlyhXDpXPnzgBs2LDBaP+OHTtSv359o22dO3fG09OTpUuXGrZdv36d33//nUGDBt323PI88cQTRrfbt2/P6dOnDbfXrFmDtbU1Y8aMMWyzsLDI9xyUVP369Wnfvr3hdtWqValTp45RDMuXL6d9+/a4u7sbPVcRERHk5ubmqyIuiJ+fn1EHahcXF4YPH87evXuJj48v0XH69++frzmxML/++ithYWGGqmoAJycnHnvsMc6cOcORI0eM9h8xYgT29vYFljV69GjDdTc3N+rUqYOjoyMDBw40bK9Tpw5ubm5Gz6Otra2hH0xubi5Xr141NJ3s2bMn33FGjRpl1Ich73XKK3Pv3r3ExMQwbty4fH128prCrl27xh9//MHAgQNJSUkxPKdXr14lMjKSEydOcP78+cKfODB6Hq5fv05SUhLt27c3irkon7mSKugzcvXqVcP3wqpVq9Dr9QwcONDofePj40OtWrXyfZaLq6DPPhTtebmdxx57zKjJsn379uTm5hIbG3vHx97pvfHXX39x9epVxowZg5XVv10rhwwZgru7e5HiA2jdujWhoaGG2zVq1KB3796sXbs2X9P1ra9T3vfx+PHjjba/8MILAIbm2KioKHJycnjqqaeM9sv7Xi/ImDFjsLS0NNz+/fffSUxMZPDgwUbvAUtLS8LDww3vAXt7e2xsbNi4cWO+pps8ee/ln3/+mezs7AL3Wb58Oa6urnTt2tXoeKGhoTg5ORmOV5SyTEk63RZBXiKSl7gUprDEplatWka3Q0JCsLCwMLRBDho0iP/973+MHj2al19+mS5dutCvXz8GDBhg+ME4ceIER48eLfQH79KlS0a3g4KC8u1jZWVF//79+fbbb8nMzMTW1pZVq1aRnZ1dpITFzs4u3/Hd3d2NPkixsbH4+vri4OBgtF/NmjXvWH5R1KhRI9+2W2M4ceIEBw4cKPJzVZCaNWsafTkD1K5dG9D6pPj4+BT7OAW9JoWJjY0lPDw83/Z69eoZ7m/YsOEdyy7oNXN1daV69er5zs/V1dXoedTr9Xz88cfMnTuXmJgYoy/8KlWq5DvWra9N3g9NXpmnTp0CMIr7VidPnkQpxRtvvMEbb7xR4D6XLl2iWrVqhZbx888/M2XKFPbt22fU9+Dm8y3KZ66kbvc8uLi4cOLECZRS+b4X8tzt6LzC3gtFeV5u506v7908Ni/pufV7wsrKqlijBQt6TmvXrk16ejqXL1/Gx8fHsP3W5yk2NhYLC4t8Mfj4+ODm5maIsbBYPTw8Ck2ubj3WiRMnAAz/cN7KxcUF0P5pmDZtGi+88ALe3t60atWKBx54gOHDhxvOpWPHjvTv35/Jkyfz0Ucf0alTJ/r06cN//vMfQwfqEydOkJSUhJeXV4HHy/uuKkpZpiQJSxG4urri6+vLgQMHbrvfgQMHqFatmuHNVphbvyDs7e3ZvHkzGzZs4JdffmHNmjUsXbqUzp07s27dOiwtLdHr9TRq1IgZM2YUWKa/v3++Mgvy8MMP89lnn/Hbb7/Rp08fli1bRt26dWnSpMltYwaM/kMwlcJiUEoZruv1erp27cpLL71U4L55icfdKu5xCntNSkNhZRf2fBXleXz33Xd54403eOSRR3j77bfx8PDAwsKCcePGFVgrUZQy7ySv3AkTJhAZGVngPrdLfv/8808efPBBOnTowNy5c/H19cXa2pqvvvrKqFNxUT5zJXWn50Gv16PT6fjtt98K3NfJyanEx4aC3wtFfV5u525e39J4b5S2wj4zRU3g7uZYee/zRYsWGSVReW6uZRo3bhy9evXi+++/Z+3atbzxxhtMnTqVP/74g2bNmqHT6VixYgXbt2/np59+Yu3atTzyyCN8+OGHbN++HScnJ/R6PV5eXnzzzTcFxpf3T01RyjIlSViK6IEHHmD+/Pls2bLFqJo+z59//smZM2d4/PHH89134sQJowz75MmT6PV6o/8cLCws6NKlC126dGHGjBm8++67vPbaa2zYsIGIiAhCQkLYv38/Xbp0uasPVIcOHfD19WXp0qW0a9eOP/74g9dee63E5d0qICCADRs2kJ6eblTLUtCIgIKUxpdFSEgIqampRERElLiMvP/0b47n77//BjC8bqVxnMIEBARw/PjxfNuPHTtmuL+srVixgvvuu48vvvjCaHtiYiKenp7FLi8kJASAQ4cOFfqc5U0JYG1tXaLndeXKldjZ2bF27Vqj/wi/+uqrfPve6TNXmLt9j4aEhKCUIigoqETJc0mOX5znxRTy3s8nT57kvvvuM2zPycnhzJkzNG7cuEjl5NVc3Ozvv//GwcHhjs2xAQEB6PV6Tpw4YajJBG0m6sTEREOMN8d68/f61atXi1TbBP9+Fry8vIr0Pg8JCeGFF17ghRde4MSJEzRt2pQPP/yQxYsXG/Zp1aoVrVq14p133uHbb79lyJAhLFmyhNGjRxMSEsL69etp27Ztkf5xul1ZpiR9WIroxRdfxN7enscff5yrV68a3Xft2jWeeOIJHBwcDEMOb5Y3jDTPrFmzAOjRo4fh8bdq2rQpgKHqduDAgZw/f5758+fn2/fGjRukpaUV6TwsLCwYMGAAP/30E4sWLSInJ6fI/VeKIjIykuzsbKM49Xp9vuegMHnzE9zNLJ4DBw4kOjqatWvX5rsvMTGRnJycO5Zx4cIFVq9ebbidnJzMwoULadq0qeE/otI4TmHuv/9+du7cSXR0tGFbWloan3/+OYGBgQX2UShtlpaW+f4DXr58+R37kBSmefPmBAUFMXPmzHyvb95xvLy86NSpE5999hkXL17MV8atw2gLilmn0xk1X505cybfzLRF+cwVxtHR8a7en/369cPS0pLJkyfne36VUvm+Xwo6PhTvM1LU58VUWrRoQZUqVZg/f77R5+abb74pchIAEB0dbdQn59y5c/zwww9069btjrVm999/P0C+pTHyarV79uwJQJcuXbCysuLTTz812m/27NlFjjMyMhIXFxfefffdAvuK5L3P09PT802BERISgrOzs+F9ev369Xzvo4J+P3Jzc3n77bfzHSsnJ8fwXipKWaYkNSxFVKtWLb7++muGDBlCo0aN8s10e+XKFb777jtD5nyzmJgYHnzwQbp37050dDSLFy/mP//5j6EZ5q233mLz5s307NmTgIAALl26xNy5c6levbqhNmfYsGEsW7aMJ554gg0bNtC2bVtyc3M5duwYy5YtY+3atbRo0aJI5zJo0CBmzZrFxIkTadSokdF/E3erT58+hIWF8cILL3Dy5Enq1q3Ljz/+aPiBuNN/h02bNsXS0pJp06aRlJSEra0tnTt3LrTttSAvvvgiP/74Iw888AAjR44kNDSUtLQ0Dh48yIoVKzhz5swdawhq167No48+yq5du/D29ubLL78kISHB6D/S0jhOYV5++WW+++47evTowbPPPouHhwdff/01MTExrFy5slwmhXvggQd46623GDVqFG3atOHgwYN88803JZ4Y0cLCgk8//ZRevXrRtGlTRo0aha+vL8eOHePw4cOGxG/OnDm0a9eORo0aMWbMGIKDg0lISCA6Opq4uLh888DcrGfPnsyYMYPu3bvzn//8h0uXLjFnzhxq1qxp1KRblM9cYUJDQ1m/fj0zZszAz8+PoKCgAvsbFSYkJIQpU6bwyiuvcObMGfr06YOzszMxMTGsXr2axx577LazHud1Kn3ttdd4+OGHsba2plevXredjKyoz4up2NjYMGnSJJ555hk6d+7MwIEDOXPmDAsWLCAkJKTItUoNGzYkMjKSZ5991jDXEcDkyZPv+NgmTZowYsQIPv/8cxITE+nYsSM7d+7k66+/pk+fPoaaH29vb5577jk+/PBDw/f6/v37+e233/D09CxSrC4uLnz66acMGzaM5s2b8/DDD1O1alXOnj3LL7/8Qtu2bZk9ezZ///03Xbp0YeDAgdSvXx8rKytWr15NQkICDz/8MABff/01c+fOpW/fvoSEhJCSksL8+fNxcXExJGEdO3bk8ccfZ+rUqezbt49u3bphbW3NiRMnWL58OR9//DEDBgwoUlkmVf4Dkyq2AwcOqMGDBytfX19lbW2tfHx81ODBg43G+efJGz535MgRNWDAAOXs7Kzc3d3V2LFj1Y0bNwz7RUVFqd69eys/Pz9lY2Oj/Pz81ODBg/MNl83KylLTpk1TDRo0ULa2tsrd3V2FhoaqyZMnq6SkJMN+FDBM9mZ6vV75+/srQE2ZMiXf/YUNa3Z0dCz0HG92+fJl9Z///Ec5OzsrV1dXNXLkSLV161YFGA2lLcz8+fNVcHCwsrS0NBriHBAQoHr27Jlv/1uHUiqlDRt85ZVXVM2aNZWNjY3y9PRUbdq0UdOnTzfMOVCYvOOsXbtWNW7cWNna2qq6deuq5cuX59u3KMfJez4LG5pY2Ot16tQpNWDAAOXm5qbs7OxUWFhYvjkz8oaNFhRbYa9Zx44dVYMGDQo97zwZGRnqhRdeUL6+vsre3l61bdtWRUdHFzp09dYYCnofKaXUli1bVNeuXZWzs7NydHRUjRs3VrNmzcp37sOHD1c+Pj7K2tpaVatWTT3wwANqxYoV+eK+1RdffKFq1apleN2++uqrfO/Ton7mCnLs2DHVoUMHZW9vbzSMNe8Ytw6Vzhvaeusw5JUrV6p27dopR0dH5ejoqOrWrauefvppdfz48TvG8Pbbb6tq1aopCwsLo7Jv99kvyvOiVOHDmm8d6pr3ut88BcHdvjc++eQTFRAQoGxtbVVYWJjaunWrCg0NVd27d7/jc5J37osXLzacZ7NmzfJNkVDY66SUUtnZ2Wry5MkqKChIWVtbK39/f/XKK68YDbFXShui/8YbbygfHx9lb2+vOnfurI4ePaqqVKminnjiiTs+dzc/P5GRkcrV1VXZ2dmpkJAQNXLkSMPQ7CtXrqinn35a1a1bVzk6OipXV1cVHh6uli1bZihjz549avDgwapGjRrK1tZWeXl5qQceeMBoeHeezz//XIWGhip7e3vl7OysGjVqpF566SV14cKFYpdlCjqlTNjrSdwzvv/+e/r27cuWLVto27atqcO5rcDAQBo2bMjPP/9s6lCEuKfp9XqqVq1Kv379CmwOv5lOp+Ppp58uVtNMaUpMTMTd3Z0pU6aUar9A8S/pwyJK3Y0bN4xu5+bmMmvWLFxcXGjevLmJohJCmLOMjIx8/ScWLlzItWvXijQ1f3m69TsO/u37Ym6xVibSh0WUumeeeYYbN27QunVrMjMzWbVqFdu2bePdd98t06G9QoiKa/v27Tz//PM89NBDVKlShT179vDFF1/QsGFDwxpj5mLp0qUsWLCA+++/HycnJ7Zs2cJ3331Ht27dzL4GuSKThEWUus6dO/Phhx/y888/k5GRQc2aNZk1axZjx441dWhCCDMVGBiIv78/n3zyCdeuXcPDw4Phw4fz3nvvmd0q0I0bN8bKyor333+f5ORkQ0fcKVOmmDq0Sk36sAghhBDC7EkfFiGEEEKYPUlYhBBCCGH2KkUfFr1ez4ULF3B2di6TdSCEEEIIUfqUUqSkpODn53fHCTErRcJy4cKFfIv/CSGEEKJiOHfuHNWrV7/tPpUiYXF2dga0E77TSslCCCGEMA/Jycn4+/sbfsdvp1IkLHnNQC4uLpKwCCGEEBVMUbpzSKdbIYQQQpg9SViEEEIIYfYkYRFCCCGE2asUfViEEELcvdzcXLKzs00dhqhkrK2tsbS0vOtyJGERQoh7nFKK+Ph4EhMTTR2KqKTc3Nzw8fG5q7nSJGERQoh7XF6y4uXlhYODg0zAKUqNUor09HQuXboEgK+vb4nLkoRFCCHuYbm5uYZkpUqVKqYOR1RC9vb2AFy6dAkvL68SNw9Jp1shhLiH5fVZcXBwMHEkojLLe3/dTR8pSViEEEJIM5AoU6Xx/pKERQghhBBmTxIWIYQQAggMDGTmzJmmDkMUQhIWIYQQFYpOp7vtZdKkSSUqd9euXTz22GN3FVunTp0YN27cXZUhCiajhO4gPSuHEwmpNPF3M3UoQgghgIsXLxquL126lDfffJPjx48btjk5ORmuK6XIzc3FyurOP3dVq1Yt3UBFqZIalts4dy2dxpPWMejzaDJzck0djhBCCMDHx8dwcXV1RafTGW4fO3YMZ2dnfvvtN0JDQ7G1tWXLli2cOnWK3r174+3tjZOTEy1btmT9+vVG5d7aJKTT6fjf//5H3759cXBwoFatWvz44493FfvKlStp0KABtra2BAYG8uGHHxrdP3fuXGrVqoWdnR3e3t4MGDDAcN+KFSto1KgR9vb2VKlShYiICNLS0u4qnopEEpbbqO5uj5uDNRnZevafSzJ1OEIIUS6UUqRn5ZT7RSlVaufw8ssv895773H06FEaN25Mamoq999/P1FRUezdu5fu3bvTq1cvzp49e9tyJk+ezMCBAzlw4AD3338/Q4YM4dq1ayWKaffu3QwcOJCHH36YgwcPMmnSJN544w0WLFgAwF9//cWzzz7LW2+9xfHjx1mzZg0dOnQAtFqlwYMH88gjj3D06FE2btxIv379SvU5M3fSJHQbOp2O8OAq/HLgIttOXSEsyMPUIQkhRJm7kZ1L/TfXlvtxj7wViYNN6fwsvfXWW3Tt2tVw28PDgyZNmhhuv/3226xevZoff/yRsWPHFlrOyJEjGTx4MADvvvsun3zyCTt37qR79+7FjmnGjBl06dKFN954A4DatWtz5MgRPvjgA0aOHMnZs2dxdHTkgQcewNnZmYCAAJo1awZoCUtOTg79+vUjICAAgEaNGhU7hopMaljuoE2INvNj9KmrJo5ECCFEUbVo0cLodmpqKhMmTKBevXq4ubnh5OTE0aNH71jD0rhxY8N1R0dHXFxcDNPMF9fRo0dp27at0ba2bdty4sQJcnNz6dq1KwEBAQQHBzNs2DC++eYb0tPTAWjSpAldunShUaNGPPTQQ8yfP5/r16+XKI6KSmpY7qB1sJaw7D2bSEZ2LnbWd7/ipBBCmDN7a0uOvBVpkuOWFkdHR6PbEyZM4Pfff2f69OnUrFkTe3t7BgwYQFZW1m3Lsba2Nrqt0+nQ6/WlFufNnJ2d2bNnDxs3bmTdunW8+eabTJo0iV27duHm5sbvv//Otm3bWLduHbNmzeK1115jx44dBAUFlUk85kZqWO4gyNMRbxdbsnL17I69t7JZIcS9SafT4WBjVe6Xspxtd+vWrYwcOZK+ffvSqFEjfHx8OHPmTJkdryD16tVj69at+eKqXbu2YX0dKysrIiIieP/99zlw4ABnzpzhjz/+ALTXpW3btkyePJm9e/diY2PD6tWry/UcTElqWO5Ap9PRJsST1XvPE33qKm1repo6JCGEEMVUq1YtVq1aRa9evdDpdLzxxhtlVlNy+fJl9u3bZ7TN19eXF154gZYtW/L2228zaNAgoqOjmT17NnPnzgXg559/5vTp03To0AF3d3d+/fVX9Ho9derUYceOHURFRdGtWze8vLzYsWMHly9fpl69emVyDuZIaliKIK9ZKPq09GMRQoiKaMaMGbi7u9OmTRt69epFZGQkzZs3L5NjffvttzRr1szoMn/+fJo3b86yZctYsmQJDRs25M033+Stt95i5MiRALi5ubFq1So6d+5MvXr1mDdvHt999x0NGjTAxcWFzZs3c//991O7dm1ef/11PvzwQ3r06FEm52COdKoSjIlKTk7G1dWVpKQkXFxcSr38c9fSaf/+BqwsdOyf2A1HW6mYEkJUDhkZGcTExBAUFISdnZ2pwxGVVGHvs+L8fksNSxH4ezhQ3d2eHL1i15mSjb8XQgghRMlJwlJE0iwkhBBCmI4kLEXUWuZjEUIIIUxGEpYiyktYDp1PIjkj28TRCCGEEPcWSViKyNfVniBPR/QKdp6WfixCCCFEeZKEpRha/dOPZZs0CwkhhBDlShKWYjCsKyQdb4UQQohyJQlLMeTVsBy9mMz1tNuvPyGEEEKI0iMJSzFUdballpcTANullkUIIYQoN5KwFJM0CwkhROXQqVMnxo0bZ7gdGBjIzJkzb/sYnU7H999/f9fHLq1y7iWSsBSTzMcihBCm1atXL7p3717gfX/++Sc6nY4DBw4Uu9xdu3bx2GOP3W14RiZNmkTTpk3zbb948WKZrwO0YMEC3NzcyvQY5UkSlmIKD6qCTgcnLqVyKSXD1OEIIcQ959FHH+X3338nLi4u331fffUVLVq0oHHjxsUut2rVqjg4OJRGiHfk4+ODra1tuRyrspCEpZjcHW2o56Mt0LRd5mMRQohy98ADD1C1alUWLFhgtD01NZXly5fz6KOPcvXqVQYPHky1atVwcHCgUaNGfPfdd7ct99YmoRMnTtChQwfs7OyoX78+v//+e77H/Pe//6V27do4ODgQHBzMG2+8QXa2NrnoggULmDx5Mvv370en06HT6Qwx39okdPDgQTp37oy9vT1VqlThscceIzU11XD/yJEj6dOnD9OnT8fX15cqVarw9NNPG45VEmfPnqV37944OTnh4uLCwIEDSUhIMNy/f/9+7rvvPpydnXFxcSE0NJS//voLgNjYWHr16oW7uzuOjo40aNCAX3/9tcSxFIUsO1wCrUOqcORiMtGnrvJgEz9ThyOEEKVLKchOL//jWjuATnfH3aysrBg+fDgLFizgtddeQ/fPY5YvX05ubi6DBw8mNTWV0NBQ/vvf/+Li4sIvv/zCsGHDCAkJISws7I7H0Ov19OvXD29vb3bs2EFSUpJRf5c8zs7OLFiwAD8/Pw4ePMiYMWNwdnbmpZdeYtCgQRw6dIg1a9awfv16AFxdXfOVkZaWRmRkJK1bt2bXrl1cunSJ0aNHM3bsWKOkbMOGDfj6+rJhwwZOnjzJoEGDaNq0KWPGjLnj+RR0fnnJyqZNm8jJyeHpp59m0KBBbNy4EYAhQ4bQrFkzPv30UywtLdm3bx/W1tYAPP3002RlZbF582YcHR05cuQITk5OxY6jOCRhKYHWwVX4YksM0aeumDoUIYQofdnp8K4J/hl79QLYOBZp10ceeYQPPviATZs20alTJ0BrDurfvz+urq64uroyYcIEw/7PPPMMa9euZdmyZUVKWNavX8+xY8dYu3Ytfn7ac/Huu+/m63fy+uuvG64HBgYyYcIElixZwksvvYS9vT1OTk5YWVnh4+NT6LG+/fZbMjIyWLhwIY6O2vnPnj2bXr16MW3aNLy9vQFwd3dn9uzZWFpaUrduXXr27ElUVFSJEpaoqCgOHjxITEwM/v7+ACxcuJAGDRqwa9cuWrZsydmzZ3nxxRepW7cuALVq1TI8/uzZs/Tv359GjRoBEBwcXOwYikuahEogLNgDCx2cuZrOxaQbpg5HCCHuOXXr1qVNmzZ8+eWXAJw8eZI///yTRx99FIDc3FzefvttGjVqhIeHB05OTqxdu5azZ88WqfyjR4/i7+9vSFYAWrdunW+/pUuX0rZtW3x8fHBycuL1118v8jFuPlaTJk0MyQpA27Zt0ev1HD9+3LCtQYMGWFpaGm77+vpy6dKlYh3r5mP6+/sbkhWA+vXr4+bmxtGjRwEYP348o0ePJiIigvfee49Tp04Z9n322WeZMmUKbdu2ZeLEiSXq5FxcUsNSAi521jSq5sr+uCSiT12lX/Pqpg5JCCFKj7WDVtthiuMWw6OPPsozzzzDnDlz+OqrrwgJCaFjx44AfPDBB3z88cfMnDmTRo0a4ejoyLhx48jKKr1JP6OjoxkyZAiTJ08mMjISV1dXlixZwocfflhqx7hZXnNMHp1Oh16vL5NjgTbC6T//+Q+//PILv/32GxMnTmTJkiX07duX0aNHExkZyS+//MK6deuYOnUqH374Ic8880yZxSM1LCXUKkTWFRJCVFI6ndY0U96XIvRfudnAgQOxsLDg22+/ZeHChTzyyCOG/ixbt26ld+/eDB06lCZNmhAcHMzff/9d5LLr1avHuXPnuHjxomHb9u3bjfbZtm0bAQEBvPbaa7Ro0YJatWoRGxtrtI+NjQ25ubl3PNb+/ftJS0szbNu6dSsWFhbUqVOnyDEXR975nTt3zrDtyJEjJCYmUr9+fcO22rVr8/zzz7Nu3Tr69evHV199ZbjP39+fJ554glWrVvHCCy8wf/78Mok1jyQsJdQmxBOQ+ViEEMJUnJycGDRoEK+88goXL15k5MiRhvtq1arF77//zrZt2zh69CiPP/640QiYO4mIiKB27dqMGDGC/fv38+eff/Laa68Z7VOrVi3Onj3LkiVLOHXqFJ988gmrV6822icwMJCYmBj27dvHlStXyMzMzHesIUOGYGdnx4gRIzh06BAbNmzgmWeeYdiwYYb+KyWVm5vLvn37jC5Hjx4lIiKCRo0aMWTIEPbs2cPOnTsZPnw4HTt2pEWLFty4cYOxY8eyceNGYmNj2bp1K7t27aJevXoAjBs3jrVr1xITE8OePXvYsGGD4b6yIglLCbUIcMfKQsf5xBucu2aC3vRCCCF49NFHuX79OpGRkUb9TV5//XWaN29OZGQknTp1wsfHhz59+hS5XAsLC1avXs2NGzcICwtj9OjRvPPOO0b7PPjggzz//POMHTuWpk2bsm3bNt544w2jffr370/37t257777qFq1aoFDqx0cHFi7di3Xrl2jZcuWDBgwgC5dujB79uziPRkFSE1NpVmzZkaXXr16odPp+OGHH3B3d6dDhw5EREQQHBzM0qVLAbC0tOTq1asMHz6c2rVrM3DgQHr06MHkyZMBLRF6+umnqVevHt27d6d27drMnTv3ruO9HZ1SShX3QXPmzOGDDz4gPj6eJk2aMGvWrCL1ul6yZAmDBw+md+/eRuPPlVJMnDiR+fPnk5iYSNu2bfn000+NeiTfTnJyMq6uriQlJeHi4lLc0ymxAZ9u46/Y60zr34hBLWuU23GFEKK0ZGRkEBMTQ1BQEHZ2dqYOR1RShb3PivP7XewalqVLlzJ+/HgmTpzInj17aNKkCZGRkXfsqXzmzBkmTJhA+/bt8933/vvv88knnzBv3jx27NiBo6MjkZGRZGSY90yyMk2/EEIIUT6KnbDMmDGDMWPGMGrUKOrXr8+8efNwcHAwDC0rSG5urqEn9a1jtZVSzJw5k9dff53evXvTuHFjFi5cyIULF8x+YajWwf8uhFiCiiohhBBCFFGxEpasrCx2795NRETEvwVYWBAREUF0dHShj3vrrbfw8vIyjI+/WUxMDPHx8UZlurq6Eh4eXmiZmZmZJCcnG11MoXmAOzZWFiQkZ3L6StqdHyCEEEKIEilWwnLlyhVyc3Pz9Vr29vYmPj6+wMds2bKFL774otDhTnmPK06ZU6dONcxk6OrqajTxTXmys7akeQ03QJqFhBBCiLJUpqOEUlJSGDZsGPPnz8fT07PUyn3llVdISkoyXG4eR17eWgf/M7z5tCQsQoiKS5q1RVkqjfdXsWa69fT0xNLSMt9Y9oSEhALXSTh16hRnzpyhV69ehm15s/JZWVlx/Phxw+MSEhLw9fU1KrNp06YFxmFra2s2y3K3qVmFj9bD9lNaPxZdMSc+EkIIU8qbPTU9PR17e3sTRyMqq/R0bfqPW2frLY5iJSw2NjaEhoYSFRVlGM+u1+uJiopi7Nix+favW7cuBw8eNNr2+uuvk5KSwscff4y/vz/W1tb4+PgQFRVlSFCSk5PZsWMHTz75ZMnOqhw1qe6GvbUlV9Oy+DshlTo+zqYOSQghiszS0hI3NzfDSE8HBwf5x0uUGqUU6enpXLp0CTc3N6O1kIqr2GsJjR8/nhEjRtCiRQvCwsKYOXMmaWlpjBo1CoDhw4dTrVo1pk6dip2dHQ0bNjR6vJubG4DR9nHjxjFlyhRq1apFUFAQb7zxBn5+fsWa5MdUbKwsaBHozp8nrhB96ookLEKICievprukC+kJcSdubm63XbG6KIqdsAwaNIjLly/z5ptvEh8fT9OmTVmzZo2h0+zZs2exsChe15iXXnqJtLQ0HnvsMRITE2nXrh1r1qypMJMYtQ6pwp8nrrDt1FVGtg0ydThCCFEsOp0OX19fvLy8yM7ONnU4opKxtra+q5qVPCWa6dbcmGqm2zx7z16n79xtuNpbs+eNrlhaSHWqEEIIcSdlOtOtyK9RNVecbK1IupHN0YummRNGCCGEqMwkYSkFVpYWhAV5ADIfixBCCFEWJGEpJTdP0y+EEEKI0iUJSynJWwhxZ8w1cnL1Jo5GCCGEqFwkYSkl9XxdcLW3JjUzh4Pnk0wdjhBCCFGpSMJSSiwtdITn9WORZiEhhBCiVEnCUorymoWk460QQghRuiRhKUVtQrSFEP86c52sHOnHIoQQQpQWSVhKUW1vJ6o42nAjO5f9cYmmDkcIIYSoNCRhKUU6nY5W/wxv3nZSmoWEEEKI0iIJSykz9GM5fcXEkQghhBCVhyQspSwvYdlzNpGM7FwTRyOEEEJUDpKwlLJgT0e8nG3JytGzJ/a6qcMRQgghKgVJWEqZTqejTYhM0y+EEEKUJklYyoDMxyKEEEKULklYykDrYG0+ln3nEknLzDFxNEIIIUTFJwlLGfD3sKeamz05esVf0o9FCCGEuGuSsJQBnU4nzUJCCCFEKZKEpYy0Ds5LWGQ+FiGEEOJuScJSRvJqWA6eTyI5I9vE0QghhBAVmyQsZcTPzZ7AKg7oFeyKuWbqcIQQQogKTRKWMpRXy7JN+rEIIYQQd0USljLUOkQb3iwdb4UQQoi7IwlLGWoV7AHA0fhkrqdlmTgaIYQQouKShKUMeTnbUdPLCaVgR4zUsgghhBAlJQlLGWsj87EIIYQQd00SljJmmI9FFkIUQgghSkwSljLW6p+E5e+EVC6nZJo4GiGEEKJikoSljLk72lDP1wWA7VLLIoQQQpSIJCzlQJqFhBBCiLsjCUs5kI63QgghxN2RhKUchAV7YKGDmCtpxCdlmDocIYQQosKRhKUcuNhZ07CaKwDRp2X1ZiGEEKK4JGEpJ4Z1hU5Ks5AQQghRXJKwlBPpeCuEEEKUnCQs5aRloAdWFjrirt/g3LV0U4cjhBBCVCiSsJQTR1srmvi7ATJaSAghhCguSVjKkTQLCSGEECVTooRlzpw5BAYGYmdnR3h4ODt37ix031WrVtGiRQvc3NxwdHSkadOmLFq0yGifkSNHotPpjC7du3cvSWhmrfVN87EopUwcjRBCCFFxWBX3AUuXLmX8+PHMmzeP8PBwZs6cSWRkJMePH8fLyyvf/h4eHrz22mvUrVsXGxsbfv75Z0aNGoWXlxeRkZGG/bp3785XX31luG1ra1vCUzJfoQHu2FhaEJ+cQcyVNIKrOpk6JCGEEKJCKHYNy4wZMxgzZgyjRo2ifv36zJs3DwcHB7788ssC9+/UqRN9+/alXr16hISE8Nxzz9G4cWO2bNlitJ+trS0+Pj6Gi7u7e8nOyIzZWVvSrIYbIM1CQgghRHEUK2HJyspi9+7dRERE/FuAhQURERFER0ff8fFKKaKiojh+/DgdOnQwum/jxo14eXlRp04dnnzySa5eLfwHPTMzk+TkZKNLRdFapukXQgghiq1YCcuVK1fIzc3F29vbaLu3tzfx8fGFPi4pKQknJydsbGzo2bMns2bNomvXrob7u3fvzsKFC4mKimLatGls2rSJHj16kJubW2B5U6dOxdXV1XDx9/cvzmmYVJsQT0BbuVn6sQghhBBFU+w+LCXh7OzMvn37SE1NJSoqivHjxxMcHEynTp0AePjhhw37NmrUiMaNGxMSEsLGjRvp0qVLvvJeeeUVxo8fb7idnJxcYZKWJv6u2FlbcCU1ixOXUqnt7WzqkIQQQgizV6yExdPTE0tLSxISEoy2JyQk4OPjU+jjLCwsqFmzJgBNmzbl6NGjTJ061ZCw3Co4OBhPT09OnjxZYMJia2tbYTvl2lpZ0iLAgy0nrxB96qokLEIIIUQRFKtJyMbGhtDQUKKiogzb9Ho9UVFRtG7dusjl6PV6MjMzC70/Li6Oq1ev4uvrW5zwKgzDukKnZCFEIYQQoiiK3SQ0fvx4RowYQYsWLQgLC2PmzJmkpaUxatQoAIYPH061atWYOnUqoPU3adGiBSEhIWRmZvLrr7+yaNEiPv30UwBSU1OZPHky/fv3x8fHh1OnTvHSSy9Rs2ZNo2HPlUlewrIj5hp6vcLCQmfiiIQQQgjzVuyEZdCgQVy+fJk333yT+Ph4mjZtypo1awwdcc+ePYuFxb8VN2lpaTz11FPExcVhb29P3bp1Wbx4MYMGDQLA0tKSAwcO8PXXX5OYmIifnx/dunXj7bffrrDNPnfSqJorjjaWJKZnczQ+mQZ+rqYOSQghhDBrOlUJhqokJyfj6upKUlISLi4upg6nSEZ9tZMNxy/zes96jG4fbOpwhBBCiHJXnN9vWUvIRGQ+FiGEEKLoJGExkdbB2nwsO2OukZOrN3E0QgghhHmThMVE6vu54GJnRUpmDocuVJyZeoUQQghTkITFRCwtdIQHS7OQEEIIURSSsJhQ67yERRZCFEIIIW5LEhYTalNTS1h2xVwjK0f6sQghhBCFkYTFhGp7OePhaMON7FwOxCWaOhwhhBDCbEnCYkIWFjpaBXsA0o9FCCGEuB1JWEysdYg2vHmbJCxCCCFEoSRhMbG8jre7z14nIzvXxNEIIYQQ5kkSFhMLqepIVWdbsnL07D2baOpwhBBCCLMkCYuJ6XQ62him6b9i4miEEEII8yQJixmQ+ViEEEKI25OExQzkLYS471wi6Vk5Jo5GCCGEMD+SsJiBGh4OVHOzJztX8deZ66YORwghhDA7krCYAZ1ORytpFhJCCCEKJQmLmcjreCvzsQghhBD5ScJiJvL6sRw6n0RKRraJoxFCCCHMiyQsZsLPzZ6AKg7k6hW7zlwzdThCCCGEWZGExYwYmoVOSrOQEEIIcTNJWMyIdLwVQgghCiYJixnJm0DuyMVkEtOzTByNEEIIYT4kYTEjXi521PRyQinYflr6sQghhBB5JGExM3m1LNulWUgIIYQwkITFzLQ2LIQoCYsQQgiRRxIWM5PX8fZ4QgpXUjNNHI0QQghhHiRhMTMejjbU9XEGpFlICCGEyCMJixmSZiEhhBDCmCQsZqhNiCcgCYsQQgiRRxIWMxQW5IGFDk5fSSMhOcPU4QghhBAmJwmLGXK1t6aBnysgtSxCCCEESMJitgzrCp26YuJI7kFKmToCIYQQt5CExUy1CpF1hUzi+G/wdlXY8ZmpIxFCCHETSVjMVMtADywtdJy7doNz19JNHc69IScL1rwM+mzY8C5kppo6IiGEEP+QhMVMOdla0aT6P/1YpJalfOz5Gq6f0a5nJGq3hRBCmAVJWMxY3nws26XjbdnLSoPNH2jX/Vtpf7fN1mpdhBBCmJwkLGasdfA/87GcvoqSjqBla8c8SE0AtwAYugKcfCDlAhxcZurIhBBCIAmLWQsNcMfG0oKLSRmcuSr9WMpM+jXY8rF2vfPrYOsMrZ/Sbm+ZCXq9yUITQgihKVHCMmfOHAIDA7GzsyM8PJydO3cWuu+qVato0aIFbm5uODo60rRpUxYtWmS0j1KKN998E19fX+zt7YmIiODEiRMlCa1SsbexpGkNN0DmYylTWz+GzCTwagANB2jbQkeBrStcPQHHfzVtfEIIIYqfsCxdupTx48czceJE9uzZQ5MmTYiMjOTSpUsF7u/h4cFrr71GdHQ0Bw4cYNSoUYwaNYq1a9ca9nn//ff55JNPmDdvHjt27MDR0ZHIyEgyMmSW19bBMry5TCVf1JqDALq8CRb/fCTsXCBstHZ9ywyZm0UIIUys2AnLjBkzGDNmDKNGjaJ+/frMmzcPBwcHvvzyywL379SpE3379qVevXqEhITw3HPP0bhxY7Zs2QJotSszZ87k9ddfp3fv3jRu3JiFCxdy4cIFvv/++7s6ucqgzU0LIUo/ljKw+X3IydA62taONL4v/AmwsoPzu+HMFtPEJ4QQAihmwpKVlcXu3buJiIj4twALCyIiIoiOjr7j45VSREVFcfz4cTp06ABATEwM8fHxRmW6uroSHh5eaJmZmZkkJycbXSqrpjXcsLWy4EpqJicvybwgperqKdj9z9DliImg0xnf7+QFzYZq17d8VL6xCSGEMFKshOXKlSvk5ubi7e1ttN3b25v4+PhCH5eUlISTkxM2Njb07NmTWbNm0bVrVwDD44pT5tSpU3F1dTVc/P39i3MaFYqtlSUtAt0BaRYqdRveAZULtbpBQJuC92k9FnQWcCoKLu4v3/iEEEIYlMsoIWdnZ/bt28euXbt45513GD9+PBs3bixxea+88gpJSUmGy7lz50ovWDPUJkQb3rztpCQspebiATi0Urve+Y3C9/MIggb9tOtbZpZ5WEIIIQpmVZydPT09sbS0JCEhwWh7QkICPj4+hT7OwsKCmjVrAtC0aVOOHj3K1KlT6dSpk+FxCQkJ+Pr6GpXZtGnTAsuztbXF1ta2OKFXaK3+6Xi7PeYqer3CwkJ3h0eIO4p6S/vbcAD4Nr79vu3GwaEVcOR7uPYGeASXdXRCCCFuUawaFhsbG0JDQ4mKijJs0+v1REVF0bp16yKXo9fryczMBCAoKAgfHx+jMpOTk9mxY0exyqzMGld3xcHGksT0bI7Fp5g6nIrvzFY4+TtYWMF9r955f59GULMrKD1sm1X28QkhhMin2E1C48ePZ/78+Xz99dccPXqUJ598krS0NEaNGgXA8OHDeeWVVwz7T506ld9//53Tp09z9OhRPvzwQxYtWsTQoVpnRp1Ox7hx45gyZQo//vgjBw8eZPjw4fj5+dGnT5/SOcsKztrSgrAgDwC2nbpi4mgqOKUgarJ2vflwqBJStMe1e177u/cbSEm4/b5CCCFKXbGahAAGDRrE5cuXefPNN4mPj6dp06asWbPG0Gn27NmzWFj8mwelpaXx1FNPERcXh729PXXr1mXx4sUMGjTIsM9LL71EWloajz32GImJibRr1441a9ZgZ2dXCqdYObQOrsLG45fZfvoqo9tLk0SJ/b0Gzu0AK3vo8FLRHxfQBqqHQdxO2PEpREwqsxCFEELkp1OVYHKP5ORkXF1dSUpKwsXFxdThlIkDcYk8OHsrzrZW7H2zK1aWsqpCselzYV47uHREqzEpbtJx7FdYMhhsXeD5Q2DnWiZhCiHEvaI4v9/yq1cUZpDTNfBzxdnOipTMHA5fqLzzzpSpgyu0ZMXOFdo+V/zH1+4OVetCZjL8VfBEiUIIIcqGJCy3k5sDm96HH542dSRYWugID5Jp+kssJws2TNGutx0H9u7FL8PC4t9EJ3ouZMvSEUIIUV4kYbmdS4dh41TY982/c3aYUOubpukXxbR7ASSeBScfbcr9kmo4AFyqQ9ol2P9tqYUnhBDi9iRhuR3fJtB+gnb95+ch6bxJw8lbV2jXmWtk5+pNGkuFkpkKmz/Qrnd8EWwcSl6WlQ20Gatd3/qJ1i9GCCFEmZOE5U46vgR+zSEjCb5/AvSmSxTqeDvj7mBNelYuB+ISTRZHhbPjU61GxD0Imo+4+/KaD9ealK7HwJEf7r48IYQQdyQJy51YWkO/+WDtADGbtR8/E7Gw0BlmvZVmoSJKv6bVhAB0fl17Pe+WjeO/zUpbPjKLTtlCCFHZScJSFJ41IfId7fr6SZBw2GSh5DULbZOEpWi2fKSN6vFu9O+aQKUh7DEtiY0/AKf+KL1yhRBCFEgSlqIKHaUNa83NgpVjICfTJGHkdbzdHXudzBzpP3FbSedh5+fa9S5vaqN8SouDx7/NS1s+Kr1yhRBCFEgSlqLS6eDBWeDgqY0e+uNtk4QRUtWJqs62ZObo2Xs20SQxVBibpkFOBtRoA7W6ln75rZ/W1iM68yfE/VX65QshhDCQhKU4nLy0pAVg22ytT0s50+l0tA6WZqE7unIS9i7WrkdM1BLO0ubmD40GatellkUIIcqUJCzFVff+f5oCFKx+Em4klnsIec1C2yVhKdyGKaBytWa8Gq3K7jh5E8kd+wUu/112xxFCiHucJCwlEfkueARDchz8OqHcD5/X8XbvuevcyJJ+LPlc2AeHVwM66PxG2R7Lqy7U6Qko2PZx2R5LCCHuYZKwlIStE/T9HHSWcHC5tkZNOarh4YCfqx3ZuYq/Yq+V67ErhKi3tL+NHgKfhmV/vHbPa3/3LzX55IKiEjHhnE9CmCNJWErKvyV0eFG7/vN4SDxXbofW6XS0kmn6CxbzJ5yK0jrD3vdq+RzTvyUEtAN9NmyfWz7HFJVHdgZc3A/7voN1r8OivjC9DrzjA2tehdxsU0cohFmwMnUAFVqHCXByPZz/C75/Eob/WLpDZ2+jTYgnq/acl4UQb6YURE3WroeOBI+g8jt2u3EQuwX++grav6ANexbiZno9JMZqK4YnHNYul47A1VNaf6uCbJ8DF/fBgK/A2btcwxXC3EjCcjcsraHf5zCvnTa0dfscaPNMuRw6r+PtgbgkUjNzcLKVl5Ljv0LcLm1Ct7zar/JSMwK8G0LCIdj1P21JB3HvSr92U1JyGBKOwKWjkJ1W8P727uDVALzrg1d98G4ASXHw47MQuxU+7wgDF4J/WPmehxBmRH7l7laVEK0T7s/jtL4TwfeVS7+Jam721PBw4Oy1dHbFXOO+ul5lfkyzps/9t+9KqyfB2ad8j6/TaX1ZVj4KO+ZB67F3t8iiqBiyM+DKceMak4QjkBpf8P6WNlC1zk3JSQMtOXH2yT/03j9MS4KXDoErf8NX90OPadDikbIZpi+EmZOEpTSEjoS/18Lfv8GqMTBmA1jblflh24RU4ey1dKJPX5WE5cAyuHwM7NygzbOmiaF+Hy1pSozV5oAJf8w0cYjSl9ecY0hKitCc4xagJSNe9bXkxLsheISAZTG+dqvWhjF/wPdPwdEf4ZfxcH4P9JwO1valc25CVBCSsJSGvFlwP22tfYn98fa/aw+VodYhVViy6xzbTl0p82OZtZxM2PCudr3d82DvZpo4LK2g7bPwywuwbRa0GFU6iy2K8pV29aZmnJI05zTUhrvbOpdOPLbOWnPQ1o+1Plr7FkPCQRi0GNxqlM4xhKgAJGEpLU5V4cHZ8N0giJ6tTQUf3KlMD5k34+3hC8kkpWfj6nCP/jj+9RUknQVnX21RQlNqOgQ2vqfFc2gVNBlk2nhE4bIztFq5m2tMitOc491Au15Qc05p0+m0jt2+TWDFI9qoos86woAvIKRz2R5bCDMhCUtpqtNdWyRx91faLLhPbdP++yojXi52hFR15NTlNHbEXKVbg3Lut2EOMlNg8wfa9Y4vmb7fiLW91ocm6i1tuv5GD5XbyDFRROd3w875WkKZW8gipkbNOf/0Myluc05ZCLkPHt8ES4dpo4cW99cW9mw7Tvq1iEpPEpbSFvmOtsbQtVPa/CwDvizTL5LWIVU4dTmNbafu0YRl+6eQfkWbebjZMFNHo2nxKPz5EVw+CifWaYmsMK2cTG32452fawlLnpubc/JqTEqzOacsuNWAR9ZqTY/7FsP6SVq/lj5zzTtuIe6SJCylzcYR+s2HL7rC4VVQpwc0Hlhmh2sd7Mni7WfZfi/Ox5J2FbZ+ol3v/Lr59Bexd9P6r2z7RKtlkYTFdJLOw19fwu4FWmILWtNOg34QNgaqhVbMmglrO+g9G6qHwq8vaR1yLx+DQd9oHXWFqISkrrosVA+Fjv/Vrv8yoUxnwW0VrE1Qdiw+hauphVRvV1ZbZkBWCvg0gvp9TR2NsVZPaT+M57ZDbLSpo7m3KKXNeLx0GMxsBH9O15IVl2ra2lLPH4F+n0H1FhUzWcmj02lDnEf9Bs5+2tDn+Z3h6E+mjkyIMiEJS1lp/wJUbwmZSbD6CW2ekDJQxcmWuj5aNfD20/fQukJJcVo/BIAuk8yvn4iLLzQZrF3fOtOkodwzstK02pRP28DXD2i1DioXAttro2yeO6DNTu1U1dSRli7/llq/loB2WgK/dKjWTFRG3zlCmIqZfctXIpZW0PczsHbUpmyPnl1mh2r1z2ih6NP30PDmje9pHSYD2kLNLqaOpmBtnwN08PcabRSKKBtXT8GaV+DDevDz89poH2sHrfbhyWgY+TPU7236DrNlyckLhn8PrZ7Wbm/5SOuQm3YPNhWLSksSlrJUJQS6T9WuR70N8QfL5DBt7rWFEC//Dfu+0a53mWi+1fpVQrQfStDm0BClR6+Hv9fB4gEwq7m26GRmktb5OnIqjD8KD3ykdaa9V1haQ/d3of8XWsJ2egN83gku7DN1ZEKUCklYylrz4VDnfm0l35VjtLkfSll4UBV0Ojh1OY2E5NIv3+xsmAJKrz2vNcJNHc3ttRun/T24Aq7HmjSUSuHGdYieoyUp3z4EJ38HdFArEoashLG7ofVTpps80Bw0GgCj14N7kDYf0JeRsO9bU0clxF2ThKWs5c2C6+ilDXPNW024FLk6WNPAzwWAwfO38+nGU8QnVdLE5fweOPIDoNM6UJo7v2ba+lIqt0ybBSu9hMPw03Mwoz6sfRWux4Cdq7Zm0zO7YcgyqBVhfn2ZTMW7ATy2UUvkcjK01eR/Hg85WaaOTIgS0ymllKmDuFvJycm4urqSlJSEi4uLqcMp2N9r4dt/hjcP+16bAKoUrd4bxyurDpKRrQfAQgftalVlQGh1utX3xs7aslSPZzIL+2hV3Y0f1kZ6VASnN8LC3mBlB+MOVb5On2UlNxuO/aJ1ro7d8u92rwbaOk2NHtKmERCF0+u1iRU3TgUUVA+DgV+Di5+pIxMCKN7vtyQs5enn57VRDM6+8OQ2cPAo1eJTMrL59eBFVuyOY9eZ64btznZWPNDYjwGh1Wleww2dufb5uJPTm2Dhg2BhDc/8Be6Bpo6oaJSC+ffBhb3Q4UVtzhhRuNTL2rwpf30JKRe0bTpLqNdLW3ohoI359lsyV3+v1RZmzUjSansfWgCBbU0dlRCSsJitrDT4rANcPQkN+sKAr8rsi/fMlTRW7Ylj5Z7znE+8Ydge5OnIgNDq9G1WDT+3CrTaq1Lwvy7aLKVhj8H9H5g6ouI58gMsG66tJv38IZmR9FZK/TNl/ufajLS5/zRdOFbVVkMPHQWu1UwaYoV37TQsGaot6GhhBd3egfDHJfkTJiUJizk7vxu+6Ab6HOj7eZkvjqfXK7bHXGXF7jh+OxjPjWxtbgadDtqGeDIgtDqRDXywtzHzJqOjP2nzS1g7wnP7tGGcFYk+F+aEaclqt3egzVhTR2QesjO0GaF3fq7VQOWp3lJLTOv3Bitb08VX2WSlwY/PwqEV2u1GD0GvT0y/Bpe4Z0nCYu42faCNdLF1gSe3ltsS8amZOfx28CIr98QZTTLnZGtFz0a+9A+tTstAd/NrMtLnwtzWcOV4xW5S2bMQfnxGaxJ8bv+9/UOceE5r8tnzNaT/Mxzf0hYa9v9nyvzmpo2vMlNKW4Nr3etaZ3DvhjBokTYkXIhyJgmLucvNga96QNxObeKzET+BRfnWcJy7ls7KPXGs3BPHuWv/NhkFVHGgf/Pq9GtejeruZvJf195v4IentIXqntuvjQ6piHIy4eMmkHIRHpwNzc1kscbyohSc+RN2fAbHf9WGpgO4VIeWj2pTADh6mjbGe8mZLbB8JKRd1j5T/f4HtbuZOipxj5GEpSK4dhrmtYesVIiY/O98HeVMr1fsOnONlXvi+OXARdKy/p3Ou3VwFQaEVqd7Qx8cbU00S2h2BsxuAUnnoOvb0PZZ08RRWrbN0v6zrVILnt5R7omqSWSmwoEl2mify8f+3R7UQWv2qd2jcs9Ca86SL2h9q+J2ATro9IpWiynDw0U5kYSlotizCH4cq416GRMFvk1MGk56Vg5rDsWzck8c205dJe+d4WBjyf2NfOnfvDrhQR5YWJRjk1H0XFj7ira427N7wLoCdRQuSGYKfNRAG60xcBHUf9DUEZWdKydh13xt0rLMZG2btSM0HQwtx4BXXdPGJzQ5mdrSBn99od2u3V1bVuRennxPlBtJWCoKpbSOpMd+hqp1tYmezOQHOe56Oqv3nGflnjjOXE03bK/ubk//5tXp37w6NaqUcZNRZorWhJJ+FXp9rI0WqQyi3tZWEK4WCqOjKtcoDb0eTqzTOtGeivp3u0eIVpvSdHDFbdKr7PZ+o029kJup9WcZ9M29tbSBMIni/H6XqN5vzpw5BAYGYmdnR3h4ODt37ix03/nz59O+fXvc3d1xd3cnIiIi3/4jR45Ep9MZXbp3716S0CoWnU77IXb00qrK108ydUQG1d0deKZLLTZM6MSKJ1ozOMwfZ1sr4q7f4OOoE3T4YAMDP4tm2a5zpGbmlE0Q0XO0ZKVKTWg6tGyOYQrhT2iTyJ3frfXpqCySzsOC++G7Qf8kKzqtuWfoKhj7F7R6QpIVc9ZsCDy6Flz9tSbr/3WBQytNHZUQBsWuYVm6dCnDhw9n3rx5hIeHM3PmTJYvX87x48fx8so/1HTIkCG0bduWNm3aYGdnx7Rp01i9ejWHDx+mWjVtXoWRI0eSkJDAV199ZXicra0t7u7uRYqpwtaw5DnxO3wzQLs+bDWEdDZtPIW4kZXLuiPxrNgdx5aTVwxNRvbWlvRo6EP/0Oq0Dq5SOk1GaVe02pWsVG2SqwZ9775Mc/LLBK25JKSz9ppXdCfXw6rHtATTxhlajIQWj4JHkKkjE8WVdhVWPqLN0Aza8gcRk6WfkSgTZdokFB4eTsuWLZk9W1sXRa/X4+/vzzPPPMPLL798x8fn5ubi7u7O7NmzGT58OKAlLImJiXz//ffFCcWgwicsAL+8ALv+V2az4Ja2i0k3WL33PCt2x3H6cpphezU3e/o1r0b/5tUJ9LyLadPXvArb52j9esZsrHydAK+fgU+aa8NKH9sEfk1NHVHJ6HO1ad83TwcU+DTWpn6XIbIVmz4X/ngbtnyk3Q5sr010WZGWlcjO0D5niWe1uKvWA2s7U0clblFmCUtWVhYODg6sWLGCPn36GLaPGDGCxMREfvjhhzuWkZKSgpeXF8uXL+eBBx4AtITl+++/x8bGBnd3dzp37syUKVOoUqVKkeKqFAlLVvo/s+Ce0CbLeujrCtG3QSnF3nOJrNwdx4/7L5CS8W/zUIsAd/qHVqdnY19c7KyLXmjiOW013twsrTmhZpcyiNwMrBwDB5dptUcPLTB1NMWXkgArH/23WavFoxD5rvwoVCZHftQWTsxK1Tq+D1oE1VuYOqp/ZaVrScm103Dt1D9/T8O1GEiKA276edNZgmdt8GmozT3j0xC8G4Gzt6miF5RhwnLhwgWqVavGtm3baN26tWH7Sy+9xKZNm9ixY8cdy3jqqadYu3Ythw8fxs5O+2JbsmQJDg4OBAUFcerUKV599VWcnJyIjo7G0jL/sM/MzEwyMzONTtjf379iJyygzfT5vwhtFtw+87QOihVIRnYu648msGJ3HJv/voz+n3eWrZUFkQ186NHQB3dHG5xsrXC0tcLRxhJHWyvsrS2Nm5G+fxr2Ldb+qxvxU4VI3Eok4TB82gZ0Flofjyohpo6o6GI2w4pHIe2SNvLnwU+g0QBTRyXKwuXj2uCAK3+DpQ30eB9ajCq/42elaQnIrQnJ1VP/rjVVGBtnbWLOlAtw43rB+zh63ZTENNIuVWpJE1g5MduE5b333uP9999n48aNNG7cuND9Tp8+TUhICOvXr6dLl/z/XU+aNInJkyfn217hExbQVlb9Y4r2QXtyK7gHmDqiEklIzuD7f5qMTlxKveP+jjaWONhaUd/6Il+mP4slet72+ZgEl8Y42vyT4NhaGiU6DjZWONla4WBrqf21yftrhY1VBWlC+uYhbVRN6EitA7a50+thy4ew4V1t4jev+lptYNXapo5MlKWMZK2m5djP2u1mw+D+6aVXm5aZclMy8s/l6j9/U+Nv/1hbV6gSrDVDGi4h2l9HT+0fHqW0OWcSDkH8AYg/pF2/egqjWpg8lrbasHvvRsY1MvZF61cpis4sm4SmT5/OlClTWL9+PS1a3LlKsWrVqkyZMoXHH388332VtoYFtLbjr+6Hc9uhRmsY+UuFnlxMKcXB80ms2B3H3rOJpGXlkJaZQ3pmLqlZOdz67vvU+iN6WO5ibW4LHs8eX+Lj2lha4GBr+U+yk5fo3Hr93+Tn5u3ujjY0qe5aPksUxG7TZj22tIFxB8HZp+yPWVJpV7UVf/OGKzcdqi1CKevQ3BuUgq0zIeotLVn1a6bNJeTmX7THZyQVnJBcO63V1N2OvXv+ZCTv4uBR8lrYrDS4dNQ4iUk4rDWBFcTV/6bmpH9qZNyDKl8fu3JU5p1uw8LCmDVrFqB1uq1RowZjx44ttNPt+++/zzvvvMPatWtp1arVHY8RFxdHjRo1+P7773nwwTtPrFUp+rDc7PoZ+LSt9qHpMhHal/yH25wppcjI1pOamUN6Vg655/4i+PsHUejYHPEjCbZBhgQnLStX+5v5z9+8xCcr95/Ha3+zcvSlElu3+t7MHdIcK8sy/iJSCr6MhHM7oO046Jq/5tAsnN0Oy0dpVetW9tBzOjSrREPNRdGd+gNWPKI1sThUgQFfQnAn7b4b142bbG5OUNKv3L5chyr5kxGPYG2kWXkOQtDrIfEMxB/8N4mJPwRJZwve38ZJq2k0JDGNtflrbO5i0ME9pEwTlqVLlzJixAg+++wzwsLCmDlzJsuWLePYsWN4e3szfPhwqlWrxtSpUwGYNm0ab775Jt9++y1t27Y1lOPk5ISTkxOpqalMnjyZ/v374+Pjw6lTp3jppZdISUnh4MGD2NreeYG4SpewAOxdDD88rS0DPzqq4o4iKY6vH4SYTdDkP9D30xIVkZ2rJz0zt4BEJy/JyTXanp6VQ2pmLumZOYbE53h8Clm5egaH1eDdvg3Lvqbl+G/w3cPaYpjPHzKvuUqU0pYTWD9JG9FUpZY2Csi7gakjE6Z0PRaWDYOL+7U+WL5NtH+0CusnksfRyzgZyWvKcQ8y/5l1b1zXal/iD0HCP8nMpaPaRHv56LTzyuvY6/NP05JLtcrbJ6+Eynym29mzZ/PBBx8QHx9P06ZN+eSTTwgPDwegU6dOBAYGsmDBAgACAwOJjY3NV8bEiROZNGkSN27coE+fPuzdu5fExET8/Pzo1q0bb7/9Nt7eReu9XSkTFqW0L4SjP4FnHXh8k9nMglsmTm2ARX20ppGxf5m0786aQxd58ps9KAXPR9TmuYhaZXtAvV7rfHv5qHnVqN24Dt8/pS1UCNBwAPSaCbbOJg1LmInsG9p0DPu+Md7u5GOcjNx8qWzvndwcbWTnzUlMwiFITSh4fzu3fzv25jUtVa17T6/cLlPzVxZpV+HT1tqbP+xxuP99U0dUNpSC+fdpo6TCn4Ae00wdEYu2x/LG94cAmNqvEYPDapTtAfcvgdWPa/+Bjjtg+uT0/G5tJd/Es/+MDJkGoaPkv0NhTCmI3apNGOgRAu6BYOtk6qhML/WS1qSU15yUcEgbbaVy8+9rYaU1KVUL/fdStU6F7rtYHJKwVCYn18Pi/tr1oSuhZoRp4ykLR37QVoy1doTn9pvN5FTT1x5n9oaTWOjgs2Et6Fq/DOdryM2GT5ppq1L3nAEtHy27Y92OUtqqymtfBX229gP00Nf3RpOkEGUpO0NbgiUviYk/qNXKZCTl39faUevUXK35P5dQrcNvJfyHQRKWyubXF7XF5Jx84Klos58Ft1hyc2BuK61ateN/4b5XTR2RgVKK/648wLK/4rCztuCb0a0IDSjDYY07PoPfXtKShLG7y38eiIxk+PEZOPK9drteL+g9x7z61AhRmSil/ZNyfo9Wq3l+j1bTnJ2Wf1/HqjfVwjQHv+aV4rdAEpbKJisdPu+oTdxUr5c2lLCyZNp7FsGPY8HeQ6tdsTOv1y87V89jC/9iw/HLuDlYs+KJNtT0KqMq76w0+Kgh3Limjbxo2L9sjlOQ+INaLde101oVdbcpWvNcZXmfCVFR6HO15qPzu+HCP4lMwmFtQtFbeQRrCYzfP7Uwvo1N35xcTJKwVEYX9mmrp+pzoPdcbWXVii47Q5uCP/k8dHsH2ow1dUQFSs/KYfD8Hew/l0g1N3tWPdUGb5cymn5+4zTY+K7WKe/xP8s+YVAK9nwNv76kjXZw9dfWjPFvWbbHFUIUXfYN7Z+K87v/vVw7nX+/CtgfRhKWyurPD7VJm2yc4IktFX8l3G2zYd1r2lC/Z/aY9Ro0V1MzGTAvmpgradT1cWbZE62Ltz5SUaVf02pZstPKvs9SZir8Mh4OLNVu14qEvvMqRTWzEJVe+jWt+cjQnLS74An4rB21Pmh5fWHMrD+MJCyVlT4XFvSEs9Hg3wpG/WrWmfNtZSTDx0205o8HZ0Hz4aaO6I7OXUun36fbuJySSatgD75+JAxbqzJ4/vNWqg5sDyN/Lv3yAS4d05qArhzXFoXr8ia0eVZm7BSiolJKW/DR0JT0T3+YgmbtNaP+MJKwVGbXz8Cn7SArBTq/AR0mmDqiktnwLmyapk1E9tT2CrPQ2KHzSTz8+XZSM3Po2diXWQ83M164sTQkndeSOX02PLq+9Jtn9i+Bn5+H7HRw9tX6ywS0Kd1jCCFMT5+r9X00NCXt0UYpFdQfxj3IuCmpnPrDSMJS2e37VluIzMIKRq/Xhr9VJKmXtR/k7DQYuBDq9zZ1RMWy5cQVRi3YSXauYlTbQN58oH7pz4abt2J13Qfg4W/uvH9RZN/QRiHtWajdDu4E/f5nNsPIhRDlIDujgP4wp/Lvp7PUlhgw6g9Tt9Rr9SVhqeyUguUjtPlLqtSCxzdXrAXofnsZdnyqJVpjNphNW2px/LDvPM8t2QfAKz3q8njHkNI9wOXjMCccUPD0Tq3j3N24clJ7zyQcAnTQ6RWtdq6iNikKIUpPXn+YvKakuL8K7w8z/nCprlpdnN/vilEPL4zpdPDATDi7Q5u/5Pc3tcXoKoLrsfDXF9r1LhMrZLIC0LtpNS6nZDLll6NM/e0YVZ1t6de8eukdoGodqNsTjv0MWz+BPnNKXtahVfDjs1ozomNV6P+/fxerE0IIBw+o2UW7gPZPcfJ546akC3u1RKUUk5XikhqWiuxkFCzup10fsgJqdTVtPEWx+knY/y0EdYARP5k6mrs25ecj/G9LDFYWOr4c2ZIOtUuxeeXcLvgiAiys4bl94FrMhCgnE9a9rk06CBDQFvp/AS6+pRejEOLeoM/Vlolx8SvVYovz+y1DAiqyml20yb1AW9k57app47mTS0fhwBLtepdJJg2ltLx6fz0ebOJHjl7xxOLdHIwrYJrtkvJvqY0U0mdD9NziPfb6Gfgy8t9kpd14GP6jJCtCiJKxsCz1ZKXYIZj06OLuRUzSOkKlJsBPz8LVU5B4DlIStNV2s9K06e/NwR9TQOm12Xqrh5o6mlJhYaFj+kNNaFuzCulZuYxasJPYqwVMq11S7cZpf3cv0NqZi+LYL/BZh3+rcP+zHCImVpiRWEIIURBpEqoMLu6H+V20/8QLo7PQVt21tAVLa205c0sb7WJlc5v7/tlmaWt83bDfHR6TV37iOVg6RIvjqe1334nUzKRkZDPos+0cuZhMQBUHVj7ZBk+nUlgyXin4rL3Wq7/Tq9Dpv4Xvm5sN6ydB9GztdvWW2qy1bv53H4cQQpQBGSV0L9r3LUS9rU0SlJOpTbNujpoOvbsOpGbsUnIG/T7dRtz1GzSu7sp3Y1rhaFsKtRqHVsKKR7T1lp4/BDaO+fdJioPloyBup3a79VitU7OVzd0fXwghyogkLEL7z1yf80/ykvXvJSdLS2YM1/NuZxvvm/PPNqN9C9iv0DJvLuef645VYegqcK1m6menzJy+nEr/T7dxPT2bDrWr8sWIFlhb3mXLa24OzG4B12Og+zRo9YTx/SfWw6ox2qzBtq5aQliv190dUwghyoEkLEKY0N6z1/nP/B3cyM6lX/NqfPhQk7ufWG7XF9q6P67+8OxercktNwc2ToU//xnS7tsEHvq64q8xJYS4Z8goISFMqFkNd+YMaYalhY5Ve87z/trjd19o0yHg6AVJ57QmopR4WNTn32Sl5Wh4ZJ0kK0KISksSFiHKQOe63kzt2wiATzeeYsHWmLsr0NoOWj2pXd/wDsxrD2f+1Fbu7v8F9PzQrFe7FkKIuyUJixBlZGBLf17oWhuAyT8f4ZcDF++uwJaPgq0LJJ7Vps32agCPbYRGA+4+WCGEMHOSsAhRhsZ2rsnQVjVQCp5fuo/oU3cxuZ+d67+rczcbqi186VmrdAIVQggzJ51uhShjuXrFU9/sZu3hBJztrFj+RGvq+tzF+zQzFWydSi9AIYQwEel0K4QZsbTQ8fHDzWgZ6E5KRg4jvtzJ+cQbJS9QkhUhxD1IEhYhyoGdtSX/G96SWl5OJCRnMuLLnSSmZ5k6LCGEqDAkYRGinLg6WPP1I2H4uNhx8lIqj379FxnZuaYOSwghKgRJWIQoR35u9nz9SBgudlbsjr3OM9/tJSdXb+qwhBDC7EnCIkQ5q+PjzPzhLbCxsuD3Iwm8+eNhKkHfdyGEKFOSsAhhAuHBVfjk4abodPDtjrPM+uOkqUMSQgizJgmLECbSvaEvbz3YAIAZv//Nkp1nTRyREEKYL0lYhDChYa0Defq+EABeXX2Q9UcSTByREEKYJ0lYhDCxCd3qMCC0OnoFY7/bw56z100dkhBCmB1JWIQwMZ1Ox9R+jehUpyoZ2XoeWbCLk5dSTR2WEEKYFUlYhDAD1pYWzB3SnCbVXUlMz2bElztJSM4wdVhCCGE2JGERwkw42Fjx5ciWBFZx4HziDUZ+tYvkjGxThyWEEGZBEhYhzEgVJ1sWPhKOp5MtRy8m8/jC3WTmyGy4QgghCYsQZqZGFQcWjGqJo40l0aev8sKy/ej1MrGcEOLeJgmLEGaoYTVX5g0LxcpCx88HLjLll6MyG64Q4p4mCYsQZqp9rapMf6gJAF9ujWH+n6dNHJEQQpiOJCxCmLE+zarx6v11AXj312Os3htn4oiEEMI0SpSwzJkzh8DAQOzs7AgPD2fnzp2F7jt//nzat2+Pu7s77u7uRERE5NtfKcWbb76Jr68v9vb2REREcOLEiZKEJkSlM6Z9MI+2CwLgxeUH2Pz3ZRNHJIQQ5a/YCcvSpUsZP348EydOZM+ePTRp0oTIyEguXbpU4P4bN25k8ODBbNiwgejoaPz9/enWrRvnz5837PP+++/zySefMG/ePHbs2IGjoyORkZFkZMg8FELodDpeu78evZr4kaNXPLl4N/M3n+ZSinw+hBD3Dp0qZk++8PBwWrZsyezZswHQ6/X4+/vzzDPP8PLLL9/x8bm5ubi7uzN79myGDx+OUgo/Pz9eeOEFJkyYAEBSUhLe3t4sWLCAhx9++I5lJicn4+rqSlJSEi4uLsU5HSEqjMycXEZ9tYttp64CYGmho0MtT/qHVieinjd21pYmjlAIIYqnOL/fxaphycrKYvfu3URERPxbgIUFERERREdHF6mM9PR0srOz8fDwACAmJob4+HijMl1dXQkPDy+0zMzMTJKTk40uQlR2tlaWfDmyJW/3aUhTfzdy9YoNxy8z9tu9hL2znldXH2R37HUZTSSEqJSsirPzlStXyM3Nxdvb22i7t7c3x44dK1IZ//3vf/Hz8zMkKPHx8YYybi0z775bTZ06lcmTJxcndCEqBTtrS4a1CmBYqwBOXU5l5e44Vu89z8WkDL7dcZZvd5wl2NORfs2r0bd5daq52Zs6ZCGEKBXlOkrovffeY8mSJaxevRo7O7sSl/PKK6+QlJRkuJw7d64UoxSiYgip6sRL3euy5b+dWfxoOP2aVcPe2pLTV9KYvu5v2k37g//M387K3XGkZ+WYOlwhhLgrxaph8fT0xNLSkoSEBKPtCQkJ+Pj43Pax06dP57333mP9+vU0btzYsD3vcQkJCfj6+hqV2bRp0wLLsrW1xdbWtjihC1FpWVroaFfLk3a1PHmrTw6/HbzIyj1xbD99jW2nrrLt1FXe+OEQPRr6MiC0OuFBHlhY6EwdthBCFEuxalhsbGwIDQ0lKirKsE2v1xMVFUXr1q0Lfdz777/P22+/zZo1a2jRooXRfUFBQfj4+BiVmZyczI4dO25bphAiPydbKx5q4c+Sx1rz50v3Mb5rbQKqOJCelcvKPXEMnr+d9u9vYMa645y5kmbqcIUQosiKPUpo6dKljBgxgs8++4ywsDBmzpzJsmXLOHbsGN7e3gwfPpxq1aoxdepUAKZNm8abb77Jt99+S9u2bQ3lODk54eTkZNjnvffe4+uvvyYoKIg33niDAwcOcOTIkSI1HckoISEKp5Rid+x1Vu6J4+f9F0nJ/Ld5KDTAnf7Nq9OzsS+u9tYmjFIIcS8qzu93sRMWgNmzZ/PBBx8QHx9P06ZN+eSTTwgPDwegU6dOBAYGsmDBAgACAwOJjY3NV8bEiROZNGkSoH2hTpw4kc8//5zExETatWvH3LlzqV27dpHikYRFiKLJyM5l3ZEEVu6O488Tl8lbU9HWyoKu9b3pH1qd9jU9sbKUSbCFEGWvzBMWcyMJixDFl5Ccwfd7z7NyTxx/J6Qatns529KnWTX6N69OHR9nE0YohKjsJGERQhSZUopD55NZuSeOH/ad53p6tuG+htVc6N+8Or2bVsPD0caEUQohKiNJWIQQJZKVo2fD8Uus3B3HH8cukfNPm5GVhY776nrRv3l1Otf1wsZKmoyEEHdPEhYhxF27lpbFj/vOs3LPeQ6eTzJsd3ew5sEmfvQPrU6jaq7odDJEWghRMpKwCCFK1fH4FFbt0WbVvZSSadhey8uJ/qHV6dusGt4uJZ8MUghxb5KERQhRJnJy9Ww5eYWVe86z7nA8mTl6ACx00K5WVfo3r0ZkAx9ZiFEIUSSSsAghylzSjWx+PXiRlbvj+Cv2umF7FUcbvnusFbW9ZYSREOL2JGERQpSrM1fSWLUnjhW747iQlEHbmlVY/Gi49G8RQtxWcX6/pau/EOKuBXo6Mr5bHZY+3hobSwu2nrzKhuOXTB2WEKISkYRFCFFq/D0cGNU2EIB3fjlKdq7etAEJISoNSViEEKXqqftq4u5gzanLaSzZedbU4QghKglJWIQQpcrV3prnu2rrgH20/gTJGdl3eIQQQtyZJCxCiFI3OKwGwVUduZaWxdwNp0wdjhCiEpCERQhR6qwtLXi1Rz0Avtwaw7lr6SaOSAhR0UnCIoQoE13qedE6uApZOXreX3vc1OEIISo4SViEEGVCp9PxWs966HTw0/4L7D17/c4PEkKIQkjCIoQoMw2rudK/eXUApvxylEowT6UQwkQkYRFClKkJ3epgb23J7tjr/HYo3tThCCEqKElYhBBlysfVjjEdggF477djZObkmjgiIURFJAmLEKLMPd4hmKrOtpy9ls6i6FhThyOEqIAkYRFClDlHWysmdNMmk/sk6gTX07JMHJEQoqKRhEUIUS4GhPpT18eZ5IwcPo46YepwhBAVjCQsQohyYWmh4/We9QFYvD2W05dTTRyREKIikYRFCFFu2tXy5L46VcnRK9777ZipwxFCVCCSsAghytWr99fD0kLHuiMJbD991dThCCEqCElYhBDlqpa3Mw+39Adgyi9H0OtlMjkhxJ1JwiKEKHfPd62Nk60Vh84n8/2+86YORwhRAUjCIoQod55Otjx1XwgAH6w9zo0smUxOCHF7krAIIUzikbZBVHOz52JSBl9sOW3qcIQQZk4SFiGESdhZW/JS9zoAfLrxFJdSMkwckRDCnEnCIoQwmV6N/WhS3ZW0rFw++l0mkxNCFE4SFiGEyVhY6Hj9AW0yuaW7znI8PsXEEQkhzJUkLEIIk2oZ6EGPhj7oFbzz61FThyOEMFOSsAghTO7lHnWxttSx+e/LbPr7sqnDEUKYIUlYhBAmF1DFkeGtAwF495ej5MpkckKIW0jCIoQwC890romrvTXHE1JY9tc5U4cjhDAzkrAIIcyCm4MNz3apBcCH646Tmplj4oiEEOZEEhYhhNkY1iqAwCoOXEnNYt7GU6YORwhhRiRhEUKYDRsrC17uUQ+A+X+e5kLiDRNHJIQwFyVKWObMmUNgYCB2dnaEh4ezc+fOQvc9fPgw/fv3JzAwEJ1Ox8yZM/PtM2nSJHQ6ndGlbt26JQlNCFHBRTbwJizQg8wcPdPXHjd1OEIIM1HshGXp0qWMHz+eiRMnsmfPHpo0aUJkZCSXLl0qcP/09HSCg4N577338PHxKbTcBg0acPHiRcNly5YtxQ1NCFEJ6HQ6Xn9Aq2VZtfc8B+OSTByREMIcFDthmTFjBmPGjGHUqFHUr1+fefPm4eDgwJdfflng/i1btuSDDz7g4YcfxtbWttByrays8PHxMVw8PT2LG5oQopJoXN2NPk39AJjyyxGUkmHOQtzripWwZGVlsXv3biIiIv4twMKCiIgIoqOj7yqQEydO4OfnR3BwMEOGDOHs2bOF7puZmUlycrLRRQhRubzYvS62VhbsiLnGuiMJpg5HCGFixUpYrly5Qm5uLt7e3kbbvb29iY+PL3EQ4eHhLFiwgDVr1vDpp58SExND+/btSUkpeF2RqVOn4urqarj4+/uX+NhCCPNUzc2e0e2DAHjvt2Nk5ehNHJEQwpTMYpRQjx49eOihh2jcuDGRkZH8+uuvJCYmsmzZsgL3f+WVV0hKSjJczp2TSaaEqIye7FQTTycbYq6k8c2OWFOHI4QwoWIlLJ6enlhaWpKQYFw9m5CQcNsOtcXl5uZG7dq1OXnyZIH329ra4uLiYnQRQlQ+TrZWPN+1NgAfR50gKT3bxBEJIUylWAmLjY0NoaGhREVFGbbp9XqioqJo3bp1qQWVmprKqVOn8PX1LbUyhRAV06AW/tTyciIxPZvZG06YOhwhhIkUu0lo/PjxzJ8/n6+//pqjR4/y5JNPkpaWxqhRowAYPnw4r7zyimH/rKws9u3bx759+8jKyuL8+fPs27fPqPZkwoQJbNq0iTNnzrBt2zb69u2LpaUlgwcPLoVTFEJUZFaWFrzaUxvmvGDbGWKvppk4IiGEKVgV9wGDBg3i8uXLvPnmm8THx9O0aVPWrFlj6Ih79uxZLCz+zYMuXLhAs2bNDLenT5/O9OnT6dixIxs3bgQgLi6OwYMHc/XqVapWrUq7du3Yvn07VatWvcvTE0JUBp1qV6V9LU/+PHGFaWuOMXdIqKlDEkKUM52qBBMcJCcn4+rqSlJSkvRnEaKSOhafzP0f/4lewYonWtMi0MPUIQkh7lJxfr/NYpSQEELcSV0fFwa20KYwmPLLUZlMToh7jCQsQogKY3y32jjYWLLvXCI/Hbho6nCEEOVIEhYhRIXh5WzHEx1DAJj22zEysnNNHJEQorxIwiKEqFDGtA/Gx8WO84k3+GrrGVOHI4QoJ5KwCCEqFHsbS16MrAPA3A0nuZqaaeKIhKjcjsen8Mb3hxi3ZK9J45CERQhR4fRtVo2G1VxIycxh5nqZTE6I0paZk8sP+87z0LxtRM7czKLtsfy4/wIXEm+YLKZiz8MihBCmZmGh47X76zN4/na+3XmWEW0CqOnlbOqwhKjwzl1L55sdZ1n+1zmupmUBYGmho1t9b4a2CsDX1c5ksUnCIoSokFqHVCGinjfrjyYw9ddjfDGypalDEqJCytUr/jh2iW92xLLp78vkzRjg42LH4LAaPBzmj7eL6RKVPJKwCCEqrFfur8vG45eIOnaJrSev0Lamp6lDEqLCuJSSwdKd5/hu51kuJGUYtrev5cnQVgF0qeuFlaX59ByRhEUIUWGFVHViaKsAFmw7w5RfjvLzM+2wtNCZOiwhzJZSiujTV/lm+1nWHo4nR69Vp7g7WDOwhT+Dw2oQ6Olo4igLJgmLEKJCe7ZLLVbuiePoxWRW7okzzIYrhPhXUno2K/bE8c2OWE5f/ncB0dAAd4a2qkGPhr7YWVuaMMI7k4RFCFGheTja8Eznmrz76zGmrz3OA419cbCRrzYhAPafS2Tx9lh+OnCBjGw9AI42lvRpVo2hrQKo51tx1t+TT7UQosIb0SaQRdtjOXftBp9vPs24iNqmDkkIk0nPyuGn/RdYvP0sB88nGbbX9XFmaKsA+jSrhpNtxfv5r3gRCyHELWytLHm5ez2e/nYPn206zeCwGmYxqkGI8nTyUgqLt59l5Z44UjJyALCxtKBnY1+GtqpB8xru6HQVt4+XJCxCiErh/kY+hAa4szv2OtPXHueDh5qYOiQhylxWjp61h+P5Zkcs209fM2yv4eHAkPAaPNTCHw9HGxNGWHokYRFCVAo6nY7Xetaj39xtrNgTx8i2gTTwczV1WEKUibjr6Xy38yxLd8Vx5Z/lKSx00KWeNsFb+5qeWFSyEXOSsAghKo3mNdx5oLEvPx+4yLu/HmXxo+EVugpciJvl6hWb/77M4u2xbDh+iX9GJOPlbMvDYTV4uKU/fm72pg2yDEnCIoSoVP7bvS7rDiew9eRVNhy/ROe63qYOSYi7ciU1k6W7tAne4q7/u5ZP25pVGBoeQER9b6zNaIK3siIJixCiUvH3cGBU20A+23yad389RodaVc1qtk4hikIpxc6YayzecZY1hy6SnatVp7jaWzMgtDpDwmsQXNXJxFGWL0lYhBCVzlP31WT57jhOXkrlu13nGNYqwNQhCVEkyRnZrN5znm92xPJ3QqphexN/N4aG16BXEz+zn+CtrEjCIoSodFztrRkXUYs3fzjMR7//Te+mfrjYWZs6LCEKdeh8Eou3x/LDvgvcyM4FwN7akj7N/BgSHkDDatKBXBIWIUSlNDisBgu2neH05TTmbjjFyz3qmjokIYxk5+r5Yd8FFm2PZf+5RMP2Wl7aGll9m1eTRPsmkrAIISola0sLXu1Rj9EL/+LLrTEMCa+Bv4eDqcMSAqUUvx9J4L3fjnH6irauj7Wlju4NfRkaXoOwIA8Z3VYASViEEJVWl3petA6uQvTpq3yw9jifDG5m6pDEPe7Q+SSm/HLEMMmbp5MNo9oGMbCFP1WdbU0cnXmThEUIUWnlTSbXa/YWftx/gVFtA2lWw93UYYl7UHxSBh+sPc6qvXEoBTZWFoxpH8QTHUNwlmafIpGERQhRqTWs5kr/5tVZsTuOKb8cZcUTraW6XZSb9KwcPtt0ms83nzZ0pu3d1I8XI+tQ3V2aKItDEhYhRKU3oVsdfjlwkd2x1/ntUDz3N/I1dUiiksvVK1buiWP62uNcStGmzm8R4M7rD9Snqb+baYOroCRhEUJUej6udozpEMwnUSd477djdKnnha3VvTmXRWk7eSkFexsrqlXiKeGLa9vJK0z55ShHLiYD4O9hzys96tGjoY/U7t0FSViEEPeExzsE893Os5y9ls6i6FhGtw82dUgV2v5ziUxfd5w/T1xBp4MOtaoytFUAnet6YVnJFt0rqlOXU5n66zHWH00AwNnOimc712J4mwBJkEuBTimlTB3E3UpOTsbV1ZWkpCRcXFxMHY4Qwkwt23WOl1YewMXOik0v3oe7o42pQ6pw/k5I4cN1x1l7WPtRtrLQkaP/92fEz9WO/4TXYGBLf7yc7UwVZrm6npbFx1EnWLw9lhy9wtJCx9DwGjwXURsPeY/dVnF+vyVhEULcM3L1ip6f/Mmx+BRGtQ1kYq8Gpg6pwjh7NZ2Z6/9m9b7zKAUWOujXvDrPdamFXim+3XGWZX+d43p6NqAlMpENfRgaHkCr4Mo5r0hmTi4Lt8Uy648TJGfkABBRz4uXe9Sjpte9tc5PSUnCIoQQhdhy4gpDv9iBlYWOdc93uOcWkCuuhOQMZv1xgiU7zxlqUno09OGFbrWp6eVstG9Gdi6/HbrIouhY9pxNNGyv6eXEkPAa9GteHVf7ij+EVynFmkPxTP3tGGevpQNQz9eF13vWo21NTxNHV7FIwiKEELcx6qudbDh+GVd7a4a2qsGINoH3TPNFUV1Py2LeplMs2HaGzBw9AB1qV2VCt9o0ru52x8cfuZDM4h2xfL/3POlZ/66N82ATP4a2CqBR9Yq5Ns7+c4lM+eUIu85cB6Cqsy0vdqtD/9Dq92zfnbshCYsQQtzG2avpjFywk9OXtWnRbSwt6NusGmM6BOWrNbjXpGbm8MWfMfzvz9OkZGrNHC0C3JkQWYdWwVWKXV5KRjbf7z3P4u1nOZ6QYtjepLorQ1oF0KuxH/Y25t8h9XziDT5Yc4zv910AwM7agsc6hPB4h2AcbWX8SklJwiKEEHeQq9fWc5n/52l2x143bO9S14sxHYIJv8fWc8nIzmXx9ljmbjzFtbQsAOr7uvBiZB061al618+FUoq/Yq+zKDqW3w5dJDtX++lxtbdmQGh1hoTXMMvmudTMHD7deJL//RljqGnq17waL0bWwddVhnLfLUlYhBCiGHbHXuPzzadZdySBvG/EJtVdGdMhmO4NfLCytDBtgGUoO1fPit1xfLz+BPHJGQAEezoyvltt7m/oi0UZNHNcSc1k2V/n+HbHWeKu3zBsb1uzCkPDA4io7421iZ/zXL1i2V/n+HDd31xJ1SZ+Cw/y4PWe9Stsc5Y5koRFCCFK4PTlVL7YEsOK3XGG/6b9Pex5tG0QA1v642BTear+9XrFTwcu8NHvf3PmqtZx1M/VjnERtenXvFq5JGm5esXmvy+zeHssfxy/ZEgWvZxteTisBoPD/E1Si7H578u8++tRjsVrTViBVRx45f56dKvvfU/VupUHSViEEOIuXEnNZFF0LAujzxiG6braWzOsVQAj2gRW6FV1lVJEHb3E9HXHDT/IVRxtGNu5JoPDamBnbZr+JHHX0/lu51mW7jrHlVStScrSQkeXul4Max1A2xDPMqntudmJhBTe+fUoG49fBrTX/LkutRjaKgAbq8pby2ZKZZ6wzJkzhw8++ID4+HiaNGnCrFmzCAsLK3Dfw4cP8+abb7J7925iY2P56KOPGDdu3F2VeStJWIQQZeFGVi4r9sTxvz9PE/tPLYSNlQX9mlVjdPvgCjfXxrZTV/hg7XH2/jPk2NnOisc7BDOqbZDZdBzNytGz5nA8i7fHsjPmmmF7YBUHhoQHMCC0eqlP+HclNZOPfv+bJbvOkatXWFnoGN46kGe71MTNQSZ+K0tlmrAsXbqU4cOHM2/ePMLDw5k5cybLly/n+PHjeHl55dt/165dLFu2jNDQUJ5//nn++9//5ktYilvm3ZywEEIUl9ZBN57PNp82/NiDNknYmPbBhJl5B9195xKZvvY4W05eAbQRLqPaBvF4h2Cz/kH+OyGFb7bHsmrPecOIJRsrCx5o7MvQVgE083e7q+c9IzuXr7aeYc6Gk6T+U35kA29e7lGPIE/HUjkHcXtlmrCEh4fTsmVLZs+eDYBer8ff359nnnmGl19++baPDQwMZNy4cfkSlrspEyRhEUKUD6UUu2Ov89nm06w/elMHXX83HmsfTPeGPmY1F8et0+hbW+r4T1gNnu5cs0LNO5OWmcOP+y+weHsshy8kG7bX93VhaKsAejf1K1YNkVKKnw5cZNpvxzifqHX6bVjNhdd71i/R0G1RcsX5/S5WHWBWVha7d+/mlVdeMWyzsLAgIiKC6OjoEgVbFmUKIURZ0Ol0tAj0oEWgB6cup/K/P2NYuSeO/ecSefrbPdTwcODRdkE81KK6STvo3m4afX8PB5PFVVKOtlYMDqvBwy392XcukUXbY/n5wEWOXEzm1dUHmfrrUfo2r8bQVgHU9r79PDq7Y68z5ZcjhpoyHxc7Xupehz5Nq5V5Hxlxd4r1ibpy5Qq5ubl4e3sbbff29ubYsWMlCqAkZWZmZpKZmWm4nZycXOB+QghRVkKqOjG1XyNe6FabhdvOsHB7LGevpTPxx8N8tP5vhrUKYHjr8u2gW9A0+vc38mF81/zT6FdEOp2OZjXcaVbDnTd61mfF7ji+2RHLmavpLIyOZWF0LGFBHgxtFUD3Bj5GHWXPXUtn2ppj/HzgIgAONpY80TGEMe2DK8TEdaKYCYu5mDp1KpMnTzZ1GEIIgaeTLeO71eGJTiGs2B3H//6M4ey1dGb9cZLPNp+mf/PqjG4fREgZTopW0DT6HWtXZUK3OpV2zhB3RxvGdAjm0XZBbD11hcXbY1l/9BI7Y66xM+Yank42DGzhz4NN/Vi99zxfbTlDVq4enQ4GhvrzQrfaeLlUnGYxUcyExdPTE0tLSxISEoy2JyQk4OPjU6IASlLmK6+8wvjx4w23k5OT8ff3L9HxhRCiNDjYWDG8dSBDwgNYe1jroLv/XCLf7TzLdzvPElHPm8c7BtMiwL3UOugWNo3+i5F1CL9H+mJYWOhoX6sq7WtV5WLSDb7beY4lO89yKSWTuRtPMXfjKcO+bWtW4bX761PfT/o6VkTFSlhsbGwIDQ0lKiqKPn36AFoH2aioKMaOHVuiAEpSpq2tLba2FXceBCFE5WVpoeP+Rr70aOjDrjPX+fyfDrp5l6b+bjzeIZhuDUreQbesp9GvqHxd7RnftTbPdK7J+iMJLN4Ry9aTVwmp6shrPetxXx2ve/a5qQyK3SQ0fvx4RowYQYsWLQgLC2PmzJmkpaUxatQoAIYPH061atWYOnUqoHWqPXLkiOH6+fPn2bdvH05OTtSsWbNIZQohREWj0+kIC/IgLMiDk5dS+WLLaVbuOc++c4k8+c0eAqo4MLpdEANC/YvchyI7V8/yv+L4JKr8ptGviKwtLejRyJcejXy5kZWLrZWFPDeVQIkmjps9e7ZhkremTZvyySefEB4eDkCnTp0IDAxkwYIFAJw5c4agoKB8ZXTs2JGNGzcWqcw7kWHNQoiK4HJKJgujz7AwOpakG9oMuu4O1gxrHcjw1gF4OhVcc2wO0+gLURZkan4hhDBj6Vk5LP8rjv9tOc25a9o8ILZWFvQPrc7odkGGVYsLmkbf08mGp++ryX/Ca2BrJaNbRMUmCYsQQlQAObl61h5O4PPNp9gflwSATgdd63nTvaEPi7bHGk2j/0THEEa2CTSbafSFuFuSsAghRAWilGJnzDU+33yaqGOXjO6rKNPoC1ESZTbTrRBCiNKn0+kID65CeHAVTl5KYf7mGLbHXKVT7aoVbhp9IcqKJCxCCGFGano5M21AY1OHIYTZka7lQgghhDB7krAIIYQQwuxJwiKEEEIIsycJixBCCCHMniQsQgghhDB7krAIIYQQwuxJwiKEEEIIsycJixBCCCHMniQsQgghhDB7krAIIYQQwuxJwiKEEEIIsycJixBCCCHMniQsQgghhDB7krAIIYQQwuxZmTqA0qCUAiA5OdnEkQghhBCiqPJ+t/N+x2+nUiQsKSkpAPj7+5s4EiGEEEIUV0pKCq6urrfdR6eKktaYOb1ez4ULF3B2dkan05k6nCJJTk7G39+fc+fO4eLiYupwytW9eu5y3vfWecO9e+5y3nLeRaWUIiUlBT8/Pywsbt9LpVLUsFhYWFC9enVTh1EiLi4u99Qb+2b36rnLed977tVzl/O+t5T0vO9Us5JHOt0KIYQQwuxJwiKEEEIIsycJi4nY2toyceJEbG1tTR1KubtXz13O+946b7h3z13OW867LFSKTrdCCCGEqNykhkUIIYQQZk8SFiGEEEKYPUlYhBBCCGH2JGERQgghhNmThKUUzZkzh8DAQOzs7AgPD2fnzp2F7jt//nzat2+Pu7s77u7uRERE5Nt/5MiR6HQ6o0v37t3L+jSKrTjnvWDBgnznZGdnZ7SPUoo333wTX19f7O3tiYiI4MSJE2V9GiVSnHPv1KlTvnPX6XT07NnTsI+5v+abN2+mV69e+Pn5odPp+P777+/4mI0bN9K8eXNsbW2pWbMmCxYsyLdPcZ5HUynuua9atYquXbtStWpVXFxcaN26NWvXrjXaZ9KkSfle77p165bhWRRfcc9748aNBb7P4+PjjfYz99e8uOdd0GdXp9PRoEEDwz4V4fWeOnUqLVu2xNnZGS8vL/r06cPx48fv+Ljly5dTt25d7OzsaNSoEb/++qvR/aXxvS4JSylZunQp48ePZ+LEiezZs4cmTZoQGRnJpUuXCtx/48aNDB48mA0bNhAdHY2/vz/dunXj/PnzRvt1796dixcvGi7fffddeZxOkRX3vEGbDfHmc4qNjTW6//333+eTTz5h3rx57NixA0dHRyIjI8nIyCjr0ymW4p77qlWrjM770KFDWFpa8tBDDxntZ86veVpaGk2aNGHOnDlF2j8mJoaePXty3333sW/fPsaNG8fo0aONfrhL8h4yheKe++bNm+natSu//voru3fv5r777qNXr17s3bvXaL8GDRoYvd5btmwpi/BLrLjnnef48eNG5+Xl5WW4ryK85sU9748//tjofM+dO4eHh0e+z7e5v96bNm3i6aefZvv27fz+++9kZ2fTrVs30tLSCn3Mtm3bGDx4MI8++ih79+6lT58+9OnTh0OHDhn2KZXvdSVKRVhYmHr66acNt3Nzc5Wfn5+aOnVqkR6fk5OjnJ2d1ddff23YNmLECNW7d+/SDrVUFfe8v/rqK+Xq6lpoeXq9Xvn4+KgPPvjAsC0xMVHZ2tqq7777rtTiLg13+5p/9NFHytnZWaWmphq2VYTXPA+gVq9efdt9XnrpJdWgQQOjbYMGDVKRkZGG23f7PJpCUc69IPXr11eTJ0823J44caJq0qRJ6QVWxopy3hs2bFCAun79eqH7VLTXvCSv9+rVq5VOp1NnzpwxbKtor7dSSl26dEkBatOmTYXuM3DgQNWzZ0+jbeHh4erxxx9XSpXe97rUsJSCrKwsdu/eTUREhGGbhYUFERERREdHF6mM9PR0srOz8fDwMNq+ceNGvLy8qFOnDk8++SRXr14t1djvRknPOzU1lYCAAPz9/enduzeHDx823BcTE0N8fLxRma6uroSHhxf5uSwPpfGaf/HFFzz88MM4OjoabTfn17y4oqOjjZ4jgMjISMNzVBrPY0Wh1+tJSUnJ9xk/ceIEfn5+BAcHM2TIEM6ePWuiCEtX06ZN8fX1pWvXrmzdutWw/V55zb/44gsiIiIICAgw2l7RXu+kpCSAfO/bm93pc15a3+uSsJSCK1eukJubi7e3t9F2b2/vfO22hfnvf/+Ln5+f0QvavXt3Fi5cSFRUFNOmTWPTpk306NGD3NzcUo2/pEpy3nXq1OHLL7/khx9+YPHixej1etq0aUNcXByA4XF381yWh7t9zXfu3MmhQ4cYPXq00XZzf82LKz4+vsDnKDk5mRs3bpTKZ6eimD59OqmpqQwcONCwLTw8nAULFrBmzRo+/fRTYmJiaN++PSkpKSaM9O74+voyb948Vq5cycqVK/H396dTp07s2bMHKJ3vS3N34cIFfvvtt3yf74r2euv1esaNG0fbtm1p2LBhofsV9jnPez1L63u9UqzWXNG99957LFmyhI0bNxp1QH344YcN1xs1akTjxo0JCQlh48aNdOnSxRSh3rXWrVvTunVrw+02bdpQr149PvvsM95++20TRla+vvjiCxo1akRYWJjR9sr4mgv49ttvmTx5Mj/88INRX44ePXoYrjdu3Jjw8HACAgJYtmwZjz76qClCvWt16tShTp06httt2rTh1KlTfPTRRyxatMiEkZWfr7/+Gjc3N/r06WO0vaK93k8//TSHDh0ym342UsNSCjw9PbG0tCQhIcFoe0JCAj4+Prd97PTp03nvvfdYt24djRs3vu2+wcHBeHp6cvLkybuOuTTczXnnsba2plmzZoZzynvc3ZRZHu7m3NPS0liyZEmRvqDM7TUvLh8fnwKfIxcXF+zt7UvlPWTulixZwujRo1m2bFm+avNbubm5Ubt27Qr7ehcmLCzMcE6V/TVXSvHll18ybNgwbGxsbruvOb/eY8eO5eeff2bDhg1Ur179tvsW9jnPez1L63tdEpZSYGNjQ2hoKFFRUYZter2eqKgoo9qEW73//vu8/fbbrFmzhhYtWtzxOHFxcVy9ehVfX99SiftulfS8b5abm8vBgwcN5xQUFISPj49RmcnJyezYsaPIZZaHuzn35cuXk5mZydChQ+94HHN7zYurdevWRs8RwO+//254jkrjPWTOvvvuO0aNGsV3331nNHy9MKmpqZw6darCvt6F2bdvn+GcKvtrvmnTJk6ePFmkf0jM8fVWSjF27FhWr17NH3/8QVBQ0B0fc6fPeal9rxeru7Ao1JIlS5Stra1asGCBOnLkiHrssceUm5ubio+PV0opNWzYMPXyyy8b9n/vvfeUjY2NWrFihbp48aLhkpKSopRSKiUlRU2YMEFFR0ermJgYtX79etW8eXNVq1YtlZGRYZJzLEhxz3vy5Mlq7dq16tSpU2r37t3q4YcfVnZ2durw4cOGfd577z3l5uamfvjhB3XgwAHVu3dvFRQUpG7cuFHu53c7xT33PO3atVODBg3Kt70ivOYpKSlq7969au/evQpQM2bMUHv37lWxsbFKKaVefvllNWzYMMP+p0+fVg4ODurFF19UR48eVXPmzFGWlpZqzZo1hn3u9Dyai+Ke+zfffKOsrKzUnDlzjD7jiYmJhn1eeOEFtXHjRhUTE6O2bt2qIiIilKenp7p06VK5n19hinveH330kfr+++/ViRMn1MGDB9Vzzz2nLCws1Pr16w37VITXvLjnnWfo0KEqPDy8wDIrwuv95JNPKldXV7Vx40aj9216erphn1u/27Zu3aqsrKzU9OnT1dGjR9XEiROVtbW1OnjwoGGf0vhel4SlFM2aNUvVqFFD2djYqLCwMLV9+3bDfR07dlQjRoww3A4ICFBAvsvEiROVUkqlp6erbt26qapVq6r/t3c3IVHtfxzHP4M6xxELsWQYxLQSQyUVKagMpgg3QdBKKyLDaNOmpCQYMcJZTC1mI9ETROGmklrFbCrCFlNBDwQV06QFtRREg4gimu9d9Pf8OVdv99r14dh9v+DAYc7v/Ob3nTPn8OE8cAoKCqyystIOHjzoqx160kzqPnLkiNs2HA7b9u3b7dmzZ57+crmc9fb2WjgcNsdxbNu2bZbNZuernBmZSe1mZq9fvzZJdvv27Sl9LYZtPvnI6p+nyTo7OjosGo1OWaepqcmCwaCtWrXKLl++PKXfn/2OfjHT2qPR6E/bm/14xDsSiVgwGLTy8nJrb2+3kZGR+S3sb8y07tOnT9vq1autsLDQSktLbcuWLXbv3r0p/fp9m//Kf31iYsJCoZBdvHhx2j4Xw/aermZJnv12umPb4OCg1dTUWDAYtPr6ekulUp7ls3FcD/xvgAAAAL7FPSwAAMD3CCwAAMD3CCwAAMD3CCwAAMD3CCwAAMD3CCwAAMD3CCwAAMD3CCwAfktDQ0MKBAKamJhY6KEAmAUEFgAA4HsEFgAA4HsEFgBzIpfLKZFIaOXKlQqFQmpsbNSNGzck/f9yTSqVUkNDgwoLC7Vhwwa9fPnS08fNmzdVX18vx3FUVVWlZDLpWf7161cdP35cFRUVchxH1dXVunTpkqfN06dPtW7dOhUVFWnTpk3KZrNzWziAOUFgATAnEomEBgYGdP78eb169UpdXV3au3ev7t+/77bp7u5WMpnU48ePVVZWph07dujbt2+SfgSNtrY27dq1Sy9evNDJkyfV29urK1euuOvv27dPV69eVX9/vzKZjC5cuKDi4mLPOHp6epRMJvXkyRPl5+ers7NzXuoHMMt+7X2OAPDXvnz5YkVFRfbgwQPP5wcOHLDdu3e7b8K9du2au2xsbMxCoZBdv37dzMz27Nljra2tnvW7u7utrq7OzMyy2axJsjt37kw7hsnvuHv3rvtZKpUySTN6pT0Af+AMC4BZNzIyos+fP6u1tVXFxcXuNDAwoLdv37rtNm7c6M6XlpZqzZo1ymQykqRMJqOWlhZPvy0tLRoeHtb379/1/Plz5eXlKRqN/nQsDQ0N7nwkEpEkjY6O/usaAcyv/IUeAIDfz6dPnyRJqVRK5eXlnmWO43hCy68KhUL/qF1BQYE7HwgEJP24vwbA4sIZFgCzrq6uTo7j6MOHD6qurvZMFRUVbrtHjx658+Pj43rz5o1qa2slSbW1tUqn055+0+m0ampqlJeXp7Vr1yqXy3nuiQHw++IMC4BZt2TJEh07dkxdXV3K5XLavHmzPn78qHQ6raVLl6qyslKS1NfXp2XLlikcDqunp0fLly/Xzp07JUlHjx7V+vXrFY/H1d7erocPH+rMmTM6e/asJKmqqkodHR3q7OxUf3+/Ghsb9f79e42OjqqtrW2hSgcwRwgsAOZEPB5XWVmZEomE3r17p5KSEjU3NysWi7mXZE6dOqXDhw9reHhYTU1NunXrloLBoCSpublZg4ODOnHihOLxuCKRiPr6+rR//373O86dO6dYLKZDhw5pbGxMK1asUCwWW4hyAcyxgJnZQg8CwH/L0NCQtm7dqvHxcZWUlCz0cAAsAtzDAgAAfI/AAgAAfI9LQgAAwPc4wwIAAHyPwAIAAHyPwAIAAHyPwAIAAHyPwAIAAHyPwAIAAHyPwAIAAHyPwAIAAHyPwAIAAHzvD5PsOzg2x8/TAAAAAElFTkSuQmCC",
            "text/plain": [
              "<Figure size 640x480 with 1 Axes>"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        }
      ],
      "source": [
        "import matplotlib.pyplot as plt\n",
        "\n",
        "training_logs_df.plot(x=\"epoch\", y=[\"loss\", \"eval_loss\"])\n",
        "plt.title('Observing the performance as the training progresses')\n",
        "plt.legend(['Train Loss', 'Validation Loss'], loc='upper right')\n",
        "plt.show()"
      ]
    },
    {
      "attachments": {},
      "cell_type": "markdown",
      "metadata": {
        "id": "HUYiyvA0-TMd"
      },
      "source": [
        "For reference,  \n",
        "Here are the curves for when running with 5 epoches:  \n",
        "![image.png]()"
      ]
    },
    {
      "attachments": {},
      "cell_type": "markdown",
      "metadata": {
        "id": "F4Dl1Ucg9bLO"
      },
      "source": [
        "### Conclusion about training length\n",
        "As can be seen, the evaluation set's results show that cutting the training time by half, using a single epoch instead of two, would be better.  \n",
        "We chose two epoch so to let you witness that. It appears that after the first epoch the model starts to overfit the training set, thus becoming less accurate and less valuable.   \n",
        "We would suggest examining the results with more than two epochs as well.  "
      ]
    },
    {
      "attachments": {},
      "cell_type": "markdown",
      "metadata": {
        "id": "L4eRP4Nm-Amg"
      },
      "source": [
        "### Generate the DL train results: Use for Design Choices"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 104
        },
        "id": "Vf7JR3mbYjKd",
        "outputId": "67adeae6-0876-4128-f38c-faeb198d92a3"
      },
      "outputs": [
        {
          "data": {
            "text/html": [],
            "text/plain": [
              "<IPython.core.display.HTML object>"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "Results on the train set for a DL Language Model:\n",
            "----------------------------------------------------\n",
            "Baseline (dummy classifier) accuracy: 0.74\n",
            "Current model's accuracy: 0.99\n",
            "The accuracy lift is: 33 %\n"
          ]
        }
      ],
      "source": [
        "results_train_train = trainer.predict(train_train_tokenized)#\n",
        "predictions_train_train = np.argmax(results_train_train[0], axis=-1)\n",
        "\n",
        "accuracy_dl_train = np.mean(predictions_train_train == train_train_df[\"label\"])\n",
        "baseline_accuracy_dl_train = np.mean(most_frequent_class == train_train_df[\"label\"])\n",
        "accuracy_dl_lift_train = 100 * (accuracy_dl_train/baseline_accuracy_dl_train - 1)\n",
        "\n",
        "print(\"Results on the train set for a DL Language Model:\\n----------------------------------------------------\")\n",
        "print(\"Baseline (dummy classifier) accuracy:\", round(baseline_accuracy_dl_train, 2))\n",
        "print(\"Current model's accuracy:\", round(accuracy_dl_train, 2))\n",
        "print(\"The accuracy lift is:\", round(accuracy_dl_lift_train), \"%\")"
      ]
    },
    {
      "attachments": {},
      "cell_type": "markdown",
      "metadata": {
        "id": "Kx4OosWO-Rj4"
      },
      "source": [
        "### Generate the DL test results: Use for presenting performance"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 364
        },
        "id": "KANVFwsKvn2d",
        "outputId": "4b2c9821-bb84-4c06-8b2d-b55dcb81c5ea"
      },
      "outputs": [
        {
          "data": {
            "text/html": [],
            "text/plain": [
              "<IPython.core.display.HTML object>"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "Results on the test set for a DL Language Model:\n",
            "---------------------------------------------------\n",
            "Baseline (dummy classifier) accuracy: 0.75\n",
            "Current model's accuracy: 0.95\n",
            "The accuracy lift is: 26 %\n",
            "\n",
            "Confusion Matrix:\n",
            "[[1565   67]\n",
            " [  46  491]]\n",
            "\n",
            "Classification Report:\n",
            "              precision    recall  f1-score   support\n",
            "\n",
            "           0       0.97      0.96      0.97      1632\n",
            "           1       0.88      0.91      0.90       537\n",
            "\n",
            "    accuracy                           0.95      2169\n",
            "   macro avg       0.93      0.94      0.93      2169\n",
            "weighted avg       0.95      0.95      0.95      2169\n",
            "\n"
          ]
        }
      ],
      "source": [
        "results_test = trainer.predict(test_tokenized)\n",
        "predictions_test = np.argmax(results_test[0], axis=-1)\n",
        "\n",
        "accuracy_dl_test = np.mean(predictions_test == test_df[\"label\"])\n",
        "baseline_accuracy_dl_test = np.mean(most_frequent_class == test_df[\"label\"])\n",
        "accuracy_dl_lift = 100 * (accuracy_dl_test/baseline_accuracy_dl_test - 1)\n",
        "\n",
        "print(\"Results on the test set for a DL Language Model:\\n---------------------------------------------------\")\n",
        "print(\"Baseline (dummy classifier) accuracy:\", round(baseline_accuracy_dl_test, 2))\n",
        "print(\"Current model's accuracy:\", round(accuracy_dl_test, 2))\n",
        "print(\"The accuracy lift is:\", round(accuracy_dl_lift), \"%\")\n",
        "\n",
        "\n",
        "print(\"\\nConfusion Matrix:\")\n",
        "print(confusion_matrix(test_df[\"label\"], predictions_test))\n",
        "print(\"\\nClassification Report:\")\n",
        "print(classification_report(test_df[\"label\"], predictions_test))"
      ]
    },
    {
      "attachments": {},
      "cell_type": "markdown",
      "metadata": {
        "id": "K5UfgvaxJGPz"
      },
      "source": [
        "<a id='pytorch_native'></a>"
      ]
    }
  ],
  "metadata": {
    "accelerator": "GPU",
    "colab": {
      "provenance": [],
      "toc_visible": true
    },
    "gpuClass": "standard",
    "kernelspec": {
      "display_name": "Python 3",
      "name": "python3"
    },
    "language_info": {
      "name": "python"
    },
    "widgets": {
      "application/vnd.jupyter.widget-state+json": {
        "03eca2f571d74759b1453f03e28ee360": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HTMLModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_87b77cd65ec74d769cfaf2f056946a58",
            "placeholder": "​",
            "style": "IPY_MODEL_40c1a35ae41048a5a36aaaa5517d232e",
            "value": "Map: 100%"
          }
        },
        "040e9974eb8b4ee287b6eb8ca4536b78": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "FloatProgressModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "success",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_b288c1bf38d3494b975985f204b646e3",
            "max": 231508,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_b5e951d4a051424d9c28ec4ac9a7ac57",
            "value": 231508
          }
        },
        "050e326df53f410b8030e85c138f0ddf": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HTMLModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_843f16eb13bc4d0ea2e7b694fbdaba89",
            "placeholder": "​",
            "style": "IPY_MODEL_ba78731b08e744e7a3fb8ef2f14635d6",
            "value": " 48.0/48.0 [00:00&lt;00:00, 3.00kB/s]"
          }
        },
        "098c142a3016470b981240d818e681a5": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "0aff96d59e384791a9772374e1421869": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HBoxModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_d414285c2c164ee295122978b6fb160c",
              "IPY_MODEL_cc3a62d71b984b40a89b0bdced53c44c",
              "IPY_MODEL_28ca2f694a354d809a915386bcec72b8"
            ],
            "layout": "IPY_MODEL_54a03d9092b040adbfbb04a95032cd74"
          }
        },
        "103a2c2a91ee41e8a9f2178decfa773c": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "160b3e8a0f9144dbb30aa062eb296686": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "16350733c6ca4eff8611df6d6bcc00dd": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "17bd1279c68a42e8bb9b8c391c8cfb15": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "FloatProgressModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "success",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_86f3780520254be8a69ba0c9ecc93f1c",
            "max": 4880,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_74cb7283bb5042dcb8ac3eca570cb851",
            "value": 4880
          }
        },
        "1b09e7ca7e46432e990084365804482d": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "ProgressStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "1f7bff426453491b99c5627165929011": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "20116e05c09348c8894ad804a93f8123": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "26c7c8a8544f4f0a9e0412b3bea5f5e5": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "27737dab270a42518eefb5ec03413425": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "28ca2f694a354d809a915386bcec72b8": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HTMLModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_9c4dea561e9a45f18b77488e3a2c8c38",
            "placeholder": "​",
            "style": "IPY_MODEL_caa50f95551e4abd8bb8f910d3dcc542",
            "value": " 570/570 [00:00&lt;00:00, 25.6kB/s]"
          }
        },
        "2ab539e2c1894c9d93913a03700701e3": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HTMLModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_20116e05c09348c8894ad804a93f8123",
            "placeholder": "​",
            "style": "IPY_MODEL_55561deb9e564557bfabfead3dcf43cb",
            "value": "vocab.txt: 100%"
          }
        },
        "2af5480eafce4ce5a181b119c57ae231": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HBoxModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_2cd51affbd3e4c9a9d42deae61ef8725",
              "IPY_MODEL_3ff75e0b9dd34817bcffa6c6d4d0cf66",
              "IPY_MODEL_c65103ea708c4494b075f792ec7d2578"
            ],
            "layout": "IPY_MODEL_103a2c2a91ee41e8a9f2178decfa773c"
          }
        },
        "2cd51affbd3e4c9a9d42deae61ef8725": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HTMLModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_3f81f3084468484e8f901d09c2ea5b07",
            "placeholder": "​",
            "style": "IPY_MODEL_994f3c155dc44c27a978785a7a15bbfe",
            "value": "tokenizer.json: 100%"
          }
        },
        "313bd6d39eef44b1a998bcea69cec16b": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "360ae5e4d031449681de6d5165c0cfe6": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "394e3053b6184980ad8b1cf9ac883870": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "3bf1190a3fd8497d9344dd5240b4e57d": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "3ddce4f62bc243bbae17fbc754c74932": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "3e85747f79e3418db8dcdecb8ccc9e31": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "FloatProgressModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "success",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_f9b6c024de4c4e5198d12d83a37ae755",
            "max": 48,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_1b09e7ca7e46432e990084365804482d",
            "value": 48
          }
        },
        "3f81f3084468484e8f901d09c2ea5b07": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "3ff75e0b9dd34817bcffa6c6d4d0cf66": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "FloatProgressModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "success",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_ebc84f172be540e498eab0ac92e98879",
            "max": 466062,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_664e5b569b914d7fbde69b93d761625c",
            "value": 466062
          }
        },
        "40c1a35ae41048a5a36aaaa5517d232e": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "45b4f32b3fc54e1dac70fa3a822e4452": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HTMLModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_7a03dac8569b41dd949d6ba8bac22326",
            "placeholder": "​",
            "style": "IPY_MODEL_d1ae726bcad24b579a39735109bdb4d5",
            "value": " 4880/4880 [00:01&lt;00:00, 2526.12 examples/s]"
          }
        },
        "474054f4c4b64600ab8216e104d234bf": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "48f7aaf647894e2a94ffbbeea565d246": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "ProgressStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "49f5db556509467cb64fa6fb0fb57cb3": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "4daebb75a50b4f52a2e15f333ca90609": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "4e59624846fe48edbf91475a2b7fdebf": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HBoxModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_97ab2101ef8149cebbe40aa0b7169d3a",
              "IPY_MODEL_c9482ecfd4004fa9ac8602f782f249d6",
              "IPY_MODEL_fa48b2a69f934782a8972b00a4dba8fc"
            ],
            "layout": "IPY_MODEL_160b3e8a0f9144dbb30aa062eb296686"
          }
        },
        "5120a915db65442893ffceb18b19c751": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "FloatProgressModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "success",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_5702cfd040c441c09f9f07696555b910",
            "max": 1627,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_48f7aaf647894e2a94ffbbeea565d246",
            "value": 1627
          }
        },
        "535eb7a9b346475ab686fa25ae93f2f7": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "54a03d9092b040adbfbb04a95032cd74": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "55561deb9e564557bfabfead3dcf43cb": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "5702cfd040c441c09f9f07696555b910": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "57ad49ce27e3403398e3256d9d9b8ca8": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "ProgressStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "664e5b569b914d7fbde69b93d761625c": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "ProgressStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "694acd04ca7e4eb6b9f1ee5308ce1890": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "6d2948debccd4ca9841a0a1175120f65": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "6fac132c8e774a4d9c59c0b66e380f5d": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "7099cb7aa89f482e843d17ca5548f597": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "74cb7283bb5042dcb8ac3eca570cb851": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "ProgressStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "77bfef19bc9448a19a61c9aa816686df": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HTMLModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_bc5b7870cae843d18666503afd6aac69",
            "placeholder": "​",
            "style": "IPY_MODEL_cdfa0e4c66ab4d8995ec26bfc880ae04",
            "value": "tokenizer_config.json: 100%"
          }
        },
        "78df8e9bda114ecb9d78475ef64efd16": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "FloatProgressModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "success",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_098c142a3016470b981240d818e681a5",
            "max": 440449768,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_c9cdd34643c24241b567a053bb2fd562",
            "value": 440449768
          }
        },
        "799c054bca4f4f35a65454e255096a4d": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HTMLModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_474054f4c4b64600ab8216e104d234bf",
            "placeholder": "​",
            "style": "IPY_MODEL_3ddce4f62bc243bbae17fbc754c74932",
            "value": " 232k/232k [00:00&lt;00:00, 4.38MB/s]"
          }
        },
        "7a03dac8569b41dd949d6ba8bac22326": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "7f423470b4904dd1a01391f896dfd5d6": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HTMLModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_e98460d193cc4431abff6a1f68776b5e",
            "placeholder": "​",
            "style": "IPY_MODEL_dc6a773fa8a548bfbfe66f3036f165c0",
            "value": "Map: 100%"
          }
        },
        "810b971582fe4616b42cb0706871adef": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HBoxModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_c626d329a7894f85ab044ba5b746171d",
              "IPY_MODEL_78df8e9bda114ecb9d78475ef64efd16",
              "IPY_MODEL_c42836171b794b7683f9613593128b16"
            ],
            "layout": "IPY_MODEL_dcd4877ccf2f4115b666f3c9e82a8c54"
          }
        },
        "843f16eb13bc4d0ea2e7b694fbdaba89": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "86f3780520254be8a69ba0c9ecc93f1c": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "87b77cd65ec74d769cfaf2f056946a58": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "8e154cbae8e540fba9ad5c3ff7f86514": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "8f160746113648e4af8bc53395aedf0f": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HBoxModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_ef5d2d6ad6804c4499ca15001c945eb6",
              "IPY_MODEL_17bd1279c68a42e8bb9b8c391c8cfb15",
              "IPY_MODEL_45b4f32b3fc54e1dac70fa3a822e4452"
            ],
            "layout": "IPY_MODEL_535eb7a9b346475ab686fa25ae93f2f7"
          }
        },
        "97ab2101ef8149cebbe40aa0b7169d3a": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HTMLModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_cd9098d89fab4e519290a90133c6e1cd",
            "placeholder": "​",
            "style": "IPY_MODEL_d465ae1a0df44d85bc241da6a5810e0a",
            "value": "Downloading builder script: 100%"
          }
        },
        "98274506b9434f15a3046703f6316bc4": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "ProgressStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "994f3c155dc44c27a978785a7a15bbfe": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "99b503702cfa4282b46d92d19cfc841f": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "9c4dea561e9a45f18b77488e3a2c8c38": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "9c794b66c71e419494985a23df365b3c": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "a3cc1264338346a2b09c00d193e6fff8": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "ProgressStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "a6a6fc8a79ca4efca54e3aa635a0db16": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "b288c1bf38d3494b975985f204b646e3": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "b5843ae0eb13403780d7d5856e99cde9": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HBoxModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_03eca2f571d74759b1453f03e28ee360",
              "IPY_MODEL_f08897fd22c6466fa3217ee5bd5e3a57",
              "IPY_MODEL_ea36dc4bc5014f9c852c34a2275337f0"
            ],
            "layout": "IPY_MODEL_4daebb75a50b4f52a2e15f333ca90609"
          }
        },
        "b5e951d4a051424d9c28ec4ac9a7ac57": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "ProgressStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "b6833c3bbf5a4e9da2acafdc1c9829d8": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "ba78731b08e744e7a3fb8ef2f14635d6": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "bc5b7870cae843d18666503afd6aac69": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "c1961a2b423647e7b9e5faa3edf75db0": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "c42836171b794b7683f9613593128b16": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HTMLModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_99b503702cfa4282b46d92d19cfc841f",
            "placeholder": "​",
            "style": "IPY_MODEL_7099cb7aa89f482e843d17ca5548f597",
            "value": " 440M/440M [00:04&lt;00:00, 161MB/s]"
          }
        },
        "c626d329a7894f85ab044ba5b746171d": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HTMLModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_394e3053b6184980ad8b1cf9ac883870",
            "placeholder": "​",
            "style": "IPY_MODEL_f1eba7e9fc6d4fc892984c947cc2ec00",
            "value": "model.safetensors: 100%"
          }
        },
        "c65103ea708c4494b075f792ec7d2578": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HTMLModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_c1961a2b423647e7b9e5faa3edf75db0",
            "placeholder": "​",
            "style": "IPY_MODEL_16350733c6ca4eff8611df6d6bcc00dd",
            "value": " 466k/466k [00:00&lt;00:00, 6.25MB/s]"
          }
        },
        "c9482ecfd4004fa9ac8602f782f249d6": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "FloatProgressModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "success",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_694acd04ca7e4eb6b9f1ee5308ce1890",
            "max": 4203,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_98274506b9434f15a3046703f6316bc4",
            "value": 4203
          }
        },
        "c9cdd34643c24241b567a053bb2fd562": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "ProgressStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "caa50f95551e4abd8bb8f910d3dcc542": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "cc3a62d71b984b40a89b0bdced53c44c": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "FloatProgressModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "success",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_6fac132c8e774a4d9c59c0b66e380f5d",
            "max": 570,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_a3cc1264338346a2b09c00d193e6fff8",
            "value": 570
          }
        },
        "cd9098d89fab4e519290a90133c6e1cd": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "cdfa0e4c66ab4d8995ec26bfc880ae04": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "d1ae726bcad24b579a39735109bdb4d5": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "d414285c2c164ee295122978b6fb160c": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HTMLModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_1f7bff426453491b99c5627165929011",
            "placeholder": "​",
            "style": "IPY_MODEL_b6833c3bbf5a4e9da2acafdc1c9829d8",
            "value": "config.json: 100%"
          }
        },
        "d465ae1a0df44d85bc241da6a5810e0a": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "d579c0801d51437f808ecc022aa08597": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HBoxModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_7f423470b4904dd1a01391f896dfd5d6",
              "IPY_MODEL_5120a915db65442893ffceb18b19c751",
              "IPY_MODEL_e3285409f724447ea8d24c5303f759ae"
            ],
            "layout": "IPY_MODEL_3bf1190a3fd8497d9344dd5240b4e57d"
          }
        },
        "dc6a773fa8a548bfbfe66f3036f165c0": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "dcd4877ccf2f4115b666f3c9e82a8c54": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "dd0d83b72beb4ec1a383e52d8b9a39ec": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "e1d893a202ae491fb11ecb066723d87b": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HBoxModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_77bfef19bc9448a19a61c9aa816686df",
              "IPY_MODEL_3e85747f79e3418db8dcdecb8ccc9e31",
              "IPY_MODEL_050e326df53f410b8030e85c138f0ddf"
            ],
            "layout": "IPY_MODEL_a6a6fc8a79ca4efca54e3aa635a0db16"
          }
        },
        "e3285409f724447ea8d24c5303f759ae": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HTMLModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_27737dab270a42518eefb5ec03413425",
            "placeholder": "​",
            "style": "IPY_MODEL_9c794b66c71e419494985a23df365b3c",
            "value": " 1627/1627 [00:00&lt;00:00, 2377.11 examples/s]"
          }
        },
        "e98460d193cc4431abff6a1f68776b5e": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "ea36dc4bc5014f9c852c34a2275337f0": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HTMLModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_360ae5e4d031449681de6d5165c0cfe6",
            "placeholder": "​",
            "style": "IPY_MODEL_dd0d83b72beb4ec1a383e52d8b9a39ec",
            "value": " 2169/2169 [00:00&lt;00:00, 2469.57 examples/s]"
          }
        },
        "eb5cfe8520104bf79843ac0608b2fe0a": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HBoxModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_2ab539e2c1894c9d93913a03700701e3",
              "IPY_MODEL_040e9974eb8b4ee287b6eb8ca4536b78",
              "IPY_MODEL_799c054bca4f4f35a65454e255096a4d"
            ],
            "layout": "IPY_MODEL_49f5db556509467cb64fa6fb0fb57cb3"
          }
        },
        "ebc84f172be540e498eab0ac92e98879": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "ef5d2d6ad6804c4499ca15001c945eb6": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HTMLModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_313bd6d39eef44b1a998bcea69cec16b",
            "placeholder": "​",
            "style": "IPY_MODEL_26c7c8a8544f4f0a9e0412b3bea5f5e5",
            "value": "Map: 100%"
          }
        },
        "f08897fd22c6466fa3217ee5bd5e3a57": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "FloatProgressModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "success",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_8e154cbae8e540fba9ad5c3ff7f86514",
            "max": 2169,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_57ad49ce27e3403398e3256d9d9b8ca8",
            "value": 2169
          }
        },
        "f1eba7e9fc6d4fc892984c947cc2ec00": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "f9b6c024de4c4e5198d12d83a37ae755": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "fa48b2a69f934782a8972b00a4dba8fc": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HTMLModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_fd8f0c3b6f9944dfa21fb1613a839538",
            "placeholder": "​",
            "style": "IPY_MODEL_6d2948debccd4ca9841a0a1175120f65",
            "value": " 4.20k/4.20k [00:00&lt;00:00, 158kB/s]"
          }
        },
        "fd8f0c3b6f9944dfa21fb1613a839538": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        }
      }
    }
  },
  "nbformat": 4,
  "nbformat_minor": 0
}
